DOCVUR  BBSIHB 

M  toa  406 

Alderman,  Donald  L. 

Evaluation  of  the  TICCIT  Computer-Assisted 
Instructional  System "in  the  Community  college.  Final 
Report-  Volume  I,  ^ 
Educational  Testing  Service,  Princeton,  H.J. 
National  Science  Foundation,  Washington, 
ETS-PR-78-10 
Sep  78 
NSF-O-731 

5HHp. ;  For  related  document,  see  TH  CC8  (| 07 
HF-S1.00  HC-$28.79  Plus  Postage. 

♦Academic  Achievement;  *Achievement  Tests;  *Algebra; 
College  Hathematics;  'Community  Colleges; 
♦Composition  (Literary);  *Ccmpqter  Assisted 
Instruction;  Junior  Colleges;  Frogram  Development; 
Student  Attitudes;  Success  Factors;  leather 
Attitudes;  Testing;  lime  Factors  (learning) 


An  evaluation  of  TICCIT  (Time-shared  Interactive, 
Computer-controlled,  Information  Televisicr)   involved  over  5,000 
community  college  students  in  introductory  algebra  and  English 
composition  courses.  Comparisons  between  computer-assisted 
instruction  and  lecture- discussion  sections  cf  the  same  courses 
focused  on  four  aspects  of  student  performance:  course  ccipletion 
rates,,  achievement,  attitudes,  and  activities  (time  allocation). 
Other  evaluation  questions  focused  on  teacher  attitudes,  teacher 
role,  and  program  implementation;  including  administraticr ,  site 
management,  and  courseware  design  goals*.  It  was  decides  to  construct 
oblective  and  essay  te^ts  specifically  for  this  program,  tc  measure 
both  end-of-course  achievement  and  immediate  learning.  TICCIT  had  a 
significant  positive  impact  on  achievement  v  She  dramatic  decreases 
noted  in  course  completion  rates  may  te  inherent  in  self-faced 
instruction  because  students  who  have  trouble  managing  their  own 
instruction  are  risks.  Student  attitudes  towards  TICCIT  were  often 
less  favorable  than  toward  conventional  .teaching  methods,  but 
attitudes  improved  when  TICCIT  couses  were  supplemented  bj  small 
group  discussion  with  an  instructor-  Eesults  suggest  that  TICCIT  may 
be  inappropriate  fcr  community  colleges  since  only  those  students 
with  a  strong  initial  grasp  in  the  subject  matter  benefited 
substantially.  In  itself ,  computer  assisted  instruction  is  no 
panacea;  results  depend  on  factors  involved  in  the  instructional 
process.  (CP) 
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Summary  and  Conclusions 


[  Introduction 

The  computer  industry  we|*  through  a  period  of  tremendous  growth  between  % 
1960  and  1970,    Within  education  there  was  a  six-fold  iri&rease  in  the  number  of 
schools  with  computers  and  an  even  greater  increase  in  college  and  university 
expenditures  on  computers.     It  became  natural  to  view  computers  as  an  emerging 
technology  with  major  implications  for  education.     However,  direct  applications 
of  the  computer  for  teaching  students  accounted  for  a  scant  amount  of  the-  total 
computer  activity  in  schools  at  the  start  of  the  1970's.  '  A  major  demonstration 
of  the  effectiveness  of  computer-assisted  instruction  (CAI)  was  thought  to  be 
the  catalyst  necessary  to  promote  widespread  adoption  of  this  new  medium. 

The  MITRE  Corporation  proposed  to  develop  a  small  computer  system  which 
would  be  dedicated  to  supporting  instruction  within  a  school.    Their  TICCII 
(Time-shared ,  Interactive,  Computer-Controlled  Information  Television)  program 
was  to  bring  the  cost  of  computer-assisted  instruction  down  tp  an  affordable 
level  for  most  schools  and  thus  encourau^the  mass  dissemination  of  CAI  by 
private  industry.     In  order  to  achieve  M  Rket  success,  the  developers  of  the 
TICCIT  program  chose  community  colleges^J^heir  target  audience  and  mathematics 
and  English  courses  as  their  intended  subject  matter:     There  had  been  a  dramatic 
rise  in  student  enrollment,  at  community  colleges,  and  a  significant  proportion 
of  the  total  enrollment  occurred  in  relatively  few  mathematics  and  English 
courses.     It  seemed" that  resources  concentrated  in  selected  courses  could 
affect  large  numbers  of  community  college  students.     Under  a  contract  awarded 
by  the  National,  Science  Foundation  in  1972,  the  MITRE  Corporation  untfertook  the 
development  and  demonstration  of  the  TICCIT  program  at  community  colleges. 

The  TICCIT  program  is  a  computer-assisted  instructional  system  that 
combines  the  strengths  of  mini-computers  with  the  display  capabilities  of 
television  receivers.     By  relying  on  mini-computers  and  other  equipment  already 
available  for  purchase  commercially,  the  developers  of  the  TICCIT  program  took 
advantage  of  proven  technology  and  kept  total  system  costs  within  the  financial 
reach  of  schools  and  colleges.     The  TICCIT  systems  installed  in  community 
colleges  can  support  128  active  terminals  and  provide  instruction  equivalent  to 
five  full  courses  in  mathematics  and  English.     Options  built  into  the  TICCIT 
system  permit  students  to  exercise  control  over  their  own  instruction:  students 
choose  not  only  a  pace  at  which  to  work  but  also  a  topic  for  study,  a  difficulty 
level  appropriate  to  their  performance,  and  An  instructional  sequence  for 
learning. 

This  evaluation  covers  the  results  of  a  demonstration  of  the  TICCIT  program 
at  two  community  colleges.     It  contrasts  the  performance  of  students  in  classes 
taught  primarily  by  computer  with  the  performance  of  similar  students  from 
lecture-discussion  sections  of  the  same  courses.     Thus,  the  outcomes  of  conven- 
tional educational  practices  served  as  a  relative  standard  for  judging  the 
impact  of  the  TICCIT  program  on  student  performance.     Comparisons  focused 
on  four  aspects  of  student  performance:     course  completion  rates,  student 
achievement,  student  attitudes,  and  student  activities.     The  evaluation  also 
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documents  faculty  acceptance  of  the  TICCIT  program  and  examines  thetfole  of 
teachers  in  Courses  where  the  primary  instructional  resource  was  the  computer. 
Matters  related  to  the  cos t-e£ feet iveness  of  the  TICCIT  program  and  the  technical 
performance  of  the  computer  system  were  beyond  the  scope  of  this  evaluation. 

Implementation  of  £he  TICCIT  Program  ^ 

The  T!C6lT  program  became  an  integral  part  of  the  curriculum  at  two 
community  colleges.     Students  could  register  for  courses  and  earn  college 
credits  in  classes  offered  on  a  computer  system.    The  equivalent  of  three  full 
courses  in  algebra  and  two  courses  in  the  fundamentals  of  English  composition 
were  available  on  the  TICCIT  system.     Instructor  involvement  in  these  courses 
varied  from  direct  supervision  of  all  student  work  to  supplementary  assistance 
provided  upon  student  request.    In  English  courses,  instructors  tended  to 
choose  the  TTCCIT  lessons  appropriate  for  their  classes  and  to  take  an  active 
role  in  assigning  and  correcting  written  exercises;  instructors  in  mathematical 
courses,  where  departmental  policy  set  the  TtCCIT  coverage  according  to  curriculum 
requirements,  had  responsibility  for  managing  classes  sometimes  three  times 
the  size  bf  usual  lecture  sections  and  for  advising^students  on  their  course 
progress*    All  TICCIT  courseware  adhered  to  the  same  instructional  design, 
relying  on  an  hierarchical  content  structure  to  organize  the  subject  matter  and 
a  rule-example-practice  paradigm  to  suggest  strategies  for  student  learning. 

Implementing  an  ambitious  and  innovatiye  program  in  schools, is  in  itself  a 
significant  accomplishment.     It  required  selecting  a  sector  of  education 
conducive  to  a  market  success  for  computer-assisted  instruction,  maintaining 
the  interest* and  commitment  of  the  schools  chosen  as  demonstration  sites,  and  • 
developing  complete  and  independent  alternatives  to  traditional  college  courses. 
Because  there  was  rapid  growth  in  student  enrollment  at  community  colleges  as 
well  as  a  concentration  of  that  enrollment  in  a  few  select  courses,  the  community, 
college  seemed  to  represent  a  viable  market  for  computer-assisted  Instruction. 
By  focusing  on  introductory  mathematics  and  English  courses,  the  TICCIT  program 
could  addrpss  the  basic  deficiencies  in  student  preparation  prevalent  under 
open  admissions  at  community  colleges  and  offer  a  replacement  for  one-fifth  of 
a  college's  traditional  curriculum.    The  potential  existed  for  a  cost-effective 
application  of  computers  to  education  that  would  rekindle  commercial  interest 
in  providing  such  services. 

The  choice  of  particular  community  colleges  to  participate  in  the  project 
was  as  carefully  planned  as  the  selection  of  an  appropriate  market.     Since  the 
installation  of  a  new  computer  system  and  the  introduction  of  a  different 
curriculum  prograiff  might  initially  demand  heavy  external  support,  the  community 
*  colleges  had  to  bev  in  close  geographical  proximity  to  the  developers.  The 
colleges  also  had  to  be  able  to  accommodate  the  full  demonstration  as  shown  by 
the  physical  space  available  for  the  computer  system  and  student  carrels,  a 
curriculum  structure  parallel  to  the  garget  courses  in  the  TICCIT  program,  an 
enrollment  in  candidate  courses  high  enough  to  justify  heavy  use  of  all 
128  terminals,  and  a  student  population  similar  to  the  national  norms  in  their 
characteristics.    The  developers  of  the  TICCIT  program  sought  and  found  two  m 
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community  college*  auitibfe  to  serve  as  ekemplary  demonstration  sites  in  the 
evaluation.  •>  *  ' 

in8Ure  the  8mooth  ^orderly* transition  of  the  TICCIT  program  into  the 
college  curriculum^  the  developers  adopted  several  specific  mechanisms.  .Notable 
among  these  were:     the  appointment  of  local  project  directors  at  the  colleges 
and  of  a  central  agent  responsible  for  college  liaison  for  the  developers;  the 
establishment-  of  a ^committee  ,with  senior  representatives  of  each  participant  to 
Oversee  the  project;  the  development  of  an  implementation  plan  with  specific 
details  on  the  preparation,  installation  and  operation  of  TICCITWitfems  at 
community  colleges;  and  the  participation  of  community  college  instructors  in 
developing  content  specifications  for  .the  TICCIT  courseware.     Each  of  these 
steps  helped  to  make  the  community  colleges  active  participants  in  project 
administration  and  planning.  , 

In  order  to  develop  a  curriculum  equivalent^  five,  full  community  colleae- 
courses,  the  developers  of  the  TICCIT  courseware  took  a  systematic  approach  to 
the  design  and  productipn  of  curriculum  materials.     Each  component,  of  the 
TICCIT  curriculum,  regardless  of  content  coverage,  adhered  to  the  same  basic 
instructional  design.     That  design  depended  heavily  on  maps,  similar  to  learning 
hierarchies,;  to  l<*nd  structure  to  the  subject  matter.     A  series  of  three  maps 
.depicting  successively  finer  levels  of  detail  in  the  subject  matter  charted  the 
content  coverage  of  TICCIT  courseware.    Learner  control  "became  available  at 
the  finest  level  of  detail,  the  segment.     Each  segment  contained  a  rule  along 
with  examples  and  practice  problems  illustrating  the  application  of  that  rule. 
Through  options  -built  into  the  TICCIT  system  a  student  could  exercise  control 
over  the  sequence,  difficulty  level  and  frequency  of  rule  and  instance  displays. 

The  uniform  nature  of  this  instructional  design  across  the  TICCIT  course- 
ware invited  the  application  of  a  single  production  model  for  the  entire 
curriculum.     The  courseware  developers  assembled  production  teams  including 
members  with  different  specialty  skills,  and  [hey  defined  a  production, process 
specifying  the  flow  of  the  curriculum  along  work  stations.     This  approach  to 
curriculum  development  was  meant  to  facilitate  the  rapid  production  of  high 
quality  courseware.     It  did  encourage  contributions  from  persons  with  different 
fields  of  expertise,  and  it  did  result  in  the  completion  of  a  curricutffflf 
comparable  in  coverage  to  five  full  courses.     But  there  were  numerous ^JRges 
to  the  original  production  model  along  the  way  to  these  accomplishments.  Some 
became  necessary  because  courseware  production  took  place  concomitantly  with 
development  of  the  computer  system.     Others  occurred  because  the  production 
model  was  too  rigid  for  adapting  easily  to  schedule  demands  and  work  styles. 

The  evaluation  of  the  TICCIT  program  involved  over  5,000  students  in 
nearly  200  sections  of  target  courses.    At  least  one  academic  term  of  college 
exposure  to  a  particular  course  on  the  TICCIT  system  always  preceded  our  7 
collection  of  data  on  program  impact.    This  period  of  adjustment  enabled  /the 
colleges  to  gain  experience  with  the  TICCIT  program  and  permitted  us  ^conduct 
tield  trials  of  the  instruments  and  procedures  for  the  evaluation.  For>> 
mathematics  courses  the  demonstration  phase  of  the  project  began  with  tfe.  fall 
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^teta  of *  the  1975-76  academic  year;  it  commenced  with  tfe  start,  of  the  1976 
'.calendar  year  for  English  courses.    In  general  there  were  academic  terms        s  ; 
of  data  collectijm  on  program  results  for  each  target  course. 

7         :  *   *  \  A  •  ■'  •    '  • 

/  Results  and  Discussion       t  °    ?  . 

.  *      Course  Completion  Rates.    The  impact  of  the  Ticcfr  program  on  course  * 
completion  rates,  defined  as  the  proportion  of  students  enrolled  for  a  course 
who  later  fulfill  the  course's  requirements  and  receive  grades  with  credit,, was  - 
negative  *tn  every  case  save  one.     Of  the  17  analyses  conducted  for  target  , 
coulees  within  terms,  13  led  to  statistically  significant  and  negative  estimates 
of  treatment  effects.     The  average  completion  rate  for  mathematics  course*  Vas 

-  16Z^5lCCIT  Classes  (i.e.  ,  16  students  of  every  100  ensiled  for 'the  course  , 
recClWd  a  grade  with  credit  within  an  academic  term)  in  contrast  to,  an  average 
of  50%  for  lecture  sections.     Figures  for  English  courses  showed  an  average 
completion  rate,  of  55Z  for  TICCIT  classes  and  66Z  for  lecture^dlsc^si on 
Actions.    It"  appears  clear  that  the  TICCIT  program  had  a  detrimental  effect  on 
the  likelihdod  that  a  student  would  complete  the  college  r*quir^feats  for 
course  credit  within  an  academic  term.     Even  when  students  pursued  their  , 
studies  on  the  TICCIT  system  across  terms,  their  likelihood  of  completing  a 
course  did  not  improve.       &  i 

To  what  should  we  attribute  these  results?     It  is  common  for  developers 
to  counter  negative  findings  with  qualifications  about  the  status  of  t£ei^ 
product  \t  the  time  of  its  evaluation..  System  reliability  or  an- incomplete 
curricular  program,  for  example,  might  account  for  thtese  negative  ^^ings. 
But  the  completion  results  were  consistent  across  ac/demic  terms  as  the  TICCIT 
system  reached  a  'high  reliability  in  its  college  operations^  Results  were  also 
consistent  across  courses,  so  faults  at  just  particular  points  in  the  TICCIT 

curriculum  seenTan  inadequate  explanation  for  the  negative  fleets.     Nor is  ^ 
the  problem  the  same  as  attrition:     students  tended  to  remain  "in  their  classes  J> 
and  simply  failed  to  fulfill  the  requirements  for  receipt  of  a  grade,  with        -  j 
course  credit.  .  '  *""■ 

The  reasons  behind  the  lower  completion  rates  observed  in  TICCIT*  classes.  • 
seem  to  stem  from  fundamental  concepts  of  the  program-  itself .     Through  learner 
control  the  TICCIT  program  shifted  the  eyha^is  in  instruction  from  the  bom-, 
puter  teaching  to  the  student,  learning*   ^Stukenta. had  to  assume  responsibili^. 
for  their  own  learning.     Other  Instructional) conditions  that  allowed>tudents/-v 
to  set  their  own  pace  for  learning,  had  also  resulted  in  lower  completion 
rates  than  lecture  sections.     This  held  for  the  audiotutorial  system  thftt 
preceded  the  TICCIT  program  at  one  demonstration  site  and  the  programmed 
instruction  that  competed  with  the  TICCJT  program  at  a  second  demonstration 
site.     The  major  factor  behind  the  effects  of  the  TICCIT  program  on  course  ) 
completion  is  probably  a  generic  problem.     Programs  that  allow  each  student ^to 
proceed  at  his  or  her  own  pace  risk  losing  students  unable  to  manage; their  own 
instruction. 
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Tl^  analyses  for 'course  completion  rites  also*  call  into  q, 
the  InUial  premises  of  the  TICCIT  projecj:    That  com^nity  co 
a*  appropriate  audience*  for  the  TICC-IT  program.    The  coomunij 
other  sectors  of  education,  doubtless  has  use  for  ef£et^L±v%( 
duQtory  mathematics  and  the  fundamentals  of  Englis  ~ 
community  'colleges  continues  to  grow  even  a*/ it  d 
appropriate  target  for  a  market t success,  but  £he 
cpmmuni^y.  colleges  also  makes  it  difficult  -for  aj 
satisfy  the  range  of  studen^  abilities  and  ni^ 
strong  background  in  mathematics  actually  c^ 
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indicated  significant;  effects  to  'to  ^elleil  ptogrsii  But  these -eat imates  of 
treatment  effects  were  positive,  \$tu£i*|^ross  academic  terms  in  order  to 
increase  sample  size,  the  achieyementi  tjpts  in  mathe|iatics  courses  revealed  a 
significant  positive  effect  in  flv«M$$Pi*  cases.    These  positive  TICCIT 
effects  held  regardless  ^ihe^staU-^cal  adjustments  made  in  attempting  to 
compensate  for  non-equivalent.  group**. :  v  Students  completing  a  mathematics  course 
on  the  TICCIT  system  had  higher  post  test*  scores  than  comparable  students 
from  lecture  sections.     Estimates  of  the  size  o£  significant  TICCIT  effects 
indicated  an  increase  of  fretfeer  than  ten  percent  over  the  achievement  results 
attained  with  conventional  teaching'  practices.     The  results/from  English 
courses  were  npt^as  d^amatic.j-since  most  significant  effect*  could  be  attributed 
to  the^-particular  circumstances  ^f  the  demonstration  rather^  than  to  the  TICCIT 
program.    ,St}Al,  there  was  one  course  in  the  fundamentals  o\  English  composition 
in  which  a. significant,  positive  effect  was  replicated  across  academic  terms. 
That  effect  amounted  to  a  five  percent^ increment  in  student  achievement. 
"'     y        (      •<  ...  ':.  >■>.  .*  f,  ; 

These  impr.Qvements  in  student  achievemenf  under  the  TICCtT  program  reflect 
the  posttest  performance  of  students  who  completed  courses  on  the  computer 
system.     Estimates  of  treatment  effeSfesUep^sent  the  difference  between  the 
actual  posttest  scares  of  students  in^CCM- claases  and.  the  probable  scores  of 
those  same  students  had  they  ins tea^  been  Itf- lecture-discuVsicn  classes. 
Students  usually  took  the  posjtests  a$  thei|,f inal  examination  in  a  course  upon 
satisfying  all  other  cou.rse  requirements.;  ,TJiu«,  the  achievement  gains  reported 
for  mathematics  and  English  'courses  apply  a*4yHo  those  students  capable  of 
completing  a  course  under  the  TICCIT  program."    &iven  the  course  completion 
rates  in  TMjlT  classes,  it  should  be  clear  ,.t.h«t -this  student  (population  is 
really ^a  8uUBbu[f  of  ♦all  students, Initially  eqtVlled  for  a  course  on  the  TICCIT 
system  and  much  smaller  than  the  p^rqi^el  population  of  students  from  lecture- 
discussion  sections  of  the  same  course^.  '  In-  two  ^mathematics  courses  the 
treatment  effects  .applied  to  an  eveh  teore  sp/eclfic  student^  population,  namely 


idents  completing  algebra  courses junder  the  T«JCIT  program  in  the  evening 


divjWn  of  the  college.  \  18  would  be  inappropriate  to  infer  that  the  TICCIT  * 
program  improves  achievement  for  all  students. 

^Particularly  those  students  atronger  in  their  prior  familiarity  with  the 
^  subject  matter  benefited  from  taking  a  course  onsthe  TICCIT  system.  From 
-'pretest  scores  in  mathematics  courses  it  became  evident  that  students  weak  n 
Entrance  ability  failed  to  satisfy  course  requirements  in  TICCIT  classes  while 
studenta  stron*  in  prior  familiarity  completed  courses  under  the  TICCIT  program  J 
and  did  Well  on  posttests.     Similarly,  the  analyses  for  completion  "tes 
and  student  achievement  in  English  courses  indicated  that  the  impact  of  the 
TICCIT'  program  on  these  aspects  of  student  performance  depended  on  a  student  s 
•ntrance  ability.    The  negative  estimates  of  treatment  effects  in  ^glish 
courses  grew  larger  as  students'  pretest  scores  declined;     Since  students  with 
less  prior  familiarity  of  the  subject  matter  tend  to  require,  greater  instruc- 
tional support,  it  appears  reasonable  to  conclude-that  the  TICCIT  program 
provided  instructional  support  sufficient  for  students  of  strong  entrance 
lability  but  inadequate  for  others. 
«  j 

Subject  to  interactions  involving  different  treatment  effects  dependent 
,on  students'  prior  familiarity  with  the  subject  master,  the  TICCIT  program 

^Succeeded  in  improving  achievement  results.    Students  who  completed  courses 
under  the  TICCIT  program  generally  attained  higher  posttest  scores  than  similar 
students  in  lecture-discussion  classes.     Because  posttest  scores  included  only 
those  -items  that  reacted  objectives  common  to  both  the  TICCIT  program  and 
the  regular  curriculln,  and  because  the,  statistical  analyses  took/em°8"^^p 
characteristics  (e.g.,  age,  s employment  status)  as  well  as  prior  knowledge 
of  the  subject  matter  Into  cons»ation  whenever  sample  size  permitted,  these 
achievement  results  do  seem  to  Resent  the  positive  Impact  of  the  TICCIT 
program  itself.    Yet  the  -treatment  effects  on  student  achievement  arose 
among  students  with  a  st^nger^ initial  grasp  of  the  subject  matter.^  While  such 
students  may  constitute  k  small  percentage  of  the  community  college  s  total  - 
enrollment,  especially  for  iht^ductor^  courses  in  mathematics,  they  might  be  a 
majority  of  the  learner*.  \  another  context.    At  industrial  or  educational 

.    institutions  which  screen \nd  select  applicants  on  a  competitive  basis,  the 
TICCIT  program  could  result  in  uniform  positive  effects  on  student  achievement. 

Student  Attitudes.     The  pattern  of  results  in  the  evaluation  of  student 
attitudes  clearly  demonstrated  that  affective  reactions  trf  the  TICCIT  program 
depended  on  the  manner  of  its  use.    While  the  estimated  treatment  effects 
had  been  consistently  negative  for  course  completion  rates  and  generally 
positive  for  student  achievement,  those  for  Student  attitudes  changed  in 
direction  and  significance  as  the  context  of  TICCIT  classes  changed.  This 
aspect  of  student  performance  highlights  the  importance  of  the  conditions  under 
which  applications  of  computer-assisted  instruction  take  place,  especially  the 
role  played  by  the  teacher  in  facilitating  student  learning  on  computers. 
On  the  whole,  students  in  TICCIT  classes  reacted  less  favorably  to  their 
mathematics  course  than  did  students  of  comparable  ability  in  lecture  sections. 
English  courses  in  which  each  instructor  decided  on  the  extent  of  ^udent 
exposure  to  the  TICCIT  system  also  showed  negative  treatment  effects  on  student 
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_ j a v* j     ^      ^.      «.  AU  course  the  IICCIT  program  was 

the  primary  instructional  resource,  but  instructors  took  an  active  role  in 

i  .1     Th^Jlnfluence  of  the  context  of  a  course  on  student  attitudes  was  oarticu 

StudLr    ^  ,fr°m  8C°re8  °n  attitUde  8Calea  established  through  factor  analyses' 

.course  to  tZH  orthTtiC8  ?°Ur8e  it8elf  and  the  flt  of  instruction  i^  the 
section-      S?      9  nearly  a8  favo«bly  in  TICCIT  classes  as  in  lecture 

sections.     The  reasons  for  students'  less  positive  reactions  to  mathematics 

C°^8<8  T*"  the  TICCIT  pr08ram  8eemed  to  lle  1»  their  perceptions  oTthe 
individual  attention  devoted  to  them  and  their  judgment s'of  tn/uslf  ulnes"s  of 
specific  system  features.     When  the  studaafateacher  ratio  in  TICCIT^Usses 
^as  higher  than  that  in  lecture  sections  of  mathematics  courses  students 

^  «celvl  e88  lndlvldual  attentlon  n  TICCIT  ci™iiea»>  ^nct8IT 

program  had  not  alleviated  a  student's  need  for  the  assistance  of  an  instructor 
and  perhaps  had  generated  a  greater  demand  for  individual  attention!  Ine 

fSSS.If'S.ISSgS'  "J16  "P"8entin8  "H-ts'  reactions  to  specific 
features  of  the  TICCIT  system  support  this  conclusion:     instructors  provided 

Ticllr  T/sllT^r  °f  d1ifflcult  material  than  did  the  HELP  feature  on  the 
Si  S  J    7         8      gave  .clearer  presentations  of  mathematical  concepts  than  t** 

cSs-r        "*  /n  Engli8h  C°Ur8e8'  "P—nted  a  combinatJon'of  regular 

classroom  sessions  and  computer-assisted^  instruction,  there  were  sianif ir-nr 
negative  treatment  effects  on  every  attitude  scale.   "iS        d        ,     8  f' 
of  learning  on  the  computer  and  teaching  in  a  classroom  may  be  like 

The°TICCl¥    I"  tW°  KlnrKUCt0r8  With  dlffe"nt  approaches  to  the  subject  matter. 
The  TICCIT  program  had  been  intended  as  a  curriculum  appropriate  for  intact 
courses  and  may  best  support  such  applications. 

class«eo^ur^Hi0?  ^  th68e  1688  favorable  attitudes  among  students  in  TICCIT 
linn    \Ta         /    8  C°Ur8e  °n  the  fundamentals  of  .grammar  and  written  expres- 
sion.    Students  in  TICCIT  classes  of  this  course  expressed  a  greater  ^ree 
i  Hf%rH°nf    aati8faction  «ith  their  course  and  reporVeJg  grefter  Jttt  of 
individual  attention  had  been  devoted  to  them.     Such  attitude*  .tJRenind  the 

JoSi    "     '  P01^tJiVe  treatment  effect«  ^served  in  this  course.  Te  course 
format  apparently  drew  on  the  strengths  of  the  TICCIT  program  in  provioC 
practice  exercises,  of  the  instructors  in  elaborating  on  difficuU  mlterJal  snd 
motivating  student  work,  and  of  group  discussions  in  promoting  persona! 

P«S™'     ^  re8Ult8  POSltlVe  8tUd6nt  attl'Ude8  and  '"osttest 

studentT^rr8i8ted  ln8tructlon  by  itself  neither  guarantees  favorable 
student  attitudes  nor  meets  all  student  needs  for  further  elaboration  and 

heteioir      n  °f         8Ubj6Ct  matter*     E8Pec^ly  with  a  student  popSaUon  as 
aeveJoninn8/8  ^  8t  C°™™^  colle*e8  "  may  be  impracticaf  So  attempt 
stuLnT-      t    ,  C°mPUter  aPPllcatlon  "Pable  of  meting  the  needs  of  all 

;     Instructors  must  be  available  for  explaining  what  students  find 
difficult  to  understand.     Moreover,  a  computer  system  that  supports  the 
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equivalent  of  full  college  courses  may  relieve  instructors  of  their  usual 
duties  but  it  also  generates  new  demands  which  preclude  any^dramatic  increase 
in  no^al  student-teacher  ratios.    The  TICCIT  program  has  shown  that  computer- 
ess"™  instruction  can!  under  proper  conditions,  both  facilitate  student 
learning  and  promote  favorable  student  attitudes. 

Student  Activities.     What  differences  occurred. in  student  self-reports 
about  their  activities  seemed  a  natural  consequence  *  ^^l^L^., 
conditions  for  a  course.    Typically  exposure  to, a  course  on  the  TICCIT  system 
encouraged  a  percentage  of  students  to  take  further  courses  in  the  subject 
matter  roughly  equal  to  the  percentage  found  in  lecture-discussion  sections. 
Course  experiences  made  a  Job  related  to. the  subject  matter  seem  more attractive 
to  like  percentages  of  students  in  each  instructional  condition.  Perhaps 
unexpected  was  tie  fact  that  students  reported  as  much  interpersonal' contact  in 
555  classes  as  in  lecture-discussion  sections.     In  both  conditions  discussions 

fir.  s!^^^-^-^-^-2^-- 

So  «3T time  III  devoted  to  homework  assignments  in  lecture  sections  of 
sections  • 

Systematic  observations  of  student  interactions  with  the  TICCIT  system 

sample        »u&£«ui.  y  decrease  in  sample  size  reflected  a 

^course  throughout  the  academic  term.     This  decrease  ... 
course  tnruu^u  ttpptt  rlarfses  to  nealect  their  course  studies. 

.l1eor.  eour.e  under  the  TICCIT  program  .aide  fro.  prefeat  'core,  higher  than 

Itldllls  Vith  complete  sets  of  observations  and  others  with  fewer  observations. 

Most  of tet  rhexa  was  no  sequence  observed  for  students'  use  of  the  learner 

control  Satires  available  at  the  TICCIT  system,  primarily  due  to  the  low 

*1  4aLZ~1  nf  -tuHent  referral  to  rule  statements  and  examples.  Students 

ins  o    e  on  practice  problems.     While  the  rules  and  the  example,  of 

their  aPPUcarion  might  easily  be  presented  in  another  format,  practice  problems 

catled  the  compter  capabilities  inherent  to  the  TICCIT  system  Into  play.  It 
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colfiU*         *  PraCtice  Pro»>le»8  that  students  received  feedback  both  for 
confirming  correct  responses  and  for  providing  remediation  for  incorrect 
thTf^t^^"86^11?88  ?  practice  P"blems  evid«*  from  student  choice  of 

£    post  ive  SSSirihS^  °f  °ther  °UtCO;3:-  J'  18  consistent  with 

rhf  S! i  i:      •  .  effects  on  student  achievement,  speciftcally  those  regarding 

the  solution  of  routine  problems  in  mathematics  and,  as  shown  in"  the  ob1ec"ve 
test  of  writing  skills,  the  identification  and  correction  of  sentenfce  errors  tn 

h^h      M°nP°Kiti0n;    U8°'  PRACTICE  WaS  the  one  8*8tem  mature  that  received 
high  ratings  by  students  both  in  comparison  to  its  closest  counterpart  In 
lecture  classes  (i.e.,  homework  assignments)  and  contrasted  wUh  ratings  of 

b°    IT":?  °f  IICfT'8  l6arner  C°ntro1*     ™*  P"cti«  Problem  appeared  to 
be  the  cornerstone  of  the  TICCIT  system. 

on  TlCrCl1  8y8tem  maintained  data  °n  the  amount  of  time  spent  by  students 

exceed  llTourl  ^  "°0nt  °f  time  *n"line  f  °r  a11  "udents 

-? \    Jl  l?  6aCh  cour8e»  and  f°r  students  with  posttests  (i.e.,  those 

was  above  30  IL  °*  thei  TICCIT  the  average  'time  on-Une 

latt^  f?         T    and/ometi-»"  nearly  40  hours  per  academic  term.  These 
Ihrlt    figUre8,f0r  "udents  with  posttests  indicate  attendance  equivalent  to 
three  and  sometimes  over  four  class  periods  per  week.     That  students  in  EngUsh 

the  SiT    I*8  ?UCh  time  °n"line  38  8tudent8  in  -"hematics  courses,  despite 
the  additional  written  assignments  and  classroom  meetings  or  small  Rroup 
discussions  in  English  courses,  seemed  to  suggest  the  influent  Instructor 
contacts  on  student  attendance.     With  regular  instructor  contacts  outs We  of 

J"  °nHthe  TICC"  students  perhaps  felt  responsible  for  tneir  own 

learning  and  answerable  to  a  specific  faculty  member.  ' 

chan~~rr  R°le<     ltS  deveuloPf«  had  anticipated  that  the  TICCIT  program  would 
change  the  nature  of  a  teacher's' duties  and  responsibilities.     As  a  mainline 
application  of  computer-assisted  instruction,  the  TICCIT  program  would  become 
the  primary  resource  for  student  learning  while  instructors  gave  further 
assistance  upon  student  request.     There  were  indeed  marked  differences  between 
the  activities  reported  by  instructors  in  TICCIT  classes  a'nd  those  reported  bv 

aic8i%rUi:sr%n\KcJrVl8CU88l0?,8eC5l0n8*    °ireCt  °b8*~ati°ns  of  instructors < 
activities  in  TICCIT  classes  confirmed  the  description  given  by  instructors 

22T^iSi!,SSl.r*tM  detail  about  the  ta8k8  performed  by  teache- 

The  TICCIT  program  nearly  eliminated  the  burden  of  preparatory  work  and 
instead  enabled  instructors  to  devote  a  considerable  proportion  of  their  time 
to  individual  students.     Instructors  with  lecture-discussion  sect  ons  of  arget 

sCtuUae8nt  aUs8sUa    "  T^™^  °f  ^  time  ^ctures  and  developing 

student  assignments  and  tests.     There  was  no  such  demand  in  TICCIT  classes. 
Furthermore,  lecture  sections  of  mathematics  courses  required  one-half  of  an 
T??Jf?Ct?r  8  tlme  ,f°r  conductin8  classroom  sessions  while  didactic  work  in 
TICCIT  classes  took  only  one-fourth  of  an  instructor's  time.     This  reduction  in 
the  time  spent  on  uSual  duties  was  offset  by  increases  in  the  amount  of  time 
spent  on  other  tasks.     Specifically,  instructors  in  TICCIT  classes  devoted 
roughly  one-third  of  their  time  to  counseling  students  on  content  questions; 
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the  same  activity  accounted  for  less  than  one-tenth  of.  the  duties  performed  by 
teachers  In  lecture  sections.     Instructors  for  mathematics  courses  under  the 
TICCIT  program  spent  another  third  of-  their  time  on  monitoring  student  perform- 
ance on  mastery  tests.  ^ 

.    Responding  to  student , questions  about  the  subject  matter  and  monitoring 
student  performance  on  mastery  tests  occupied  the  bulk  of  instructor,    time  In 
TICCIT  classes.     These  activities  may  signify  the  difficulty  encountered 
by  some  students  In  learning  on  the  TICCIT  system  and,  therefore    conform  with 
the  course  completion  rates  found  under  the  TICCIT  program.     Such  e*tensive 
instructor  contact  with  Individual  students  may  also  Indicate  the  assl     ance  , 
which  enabled  other  students  both  to  complete  courses  on  the  TICCIT  system  and 
c«  attain  higher  scores  on  achievement  posttests.     These  alternative  Interpreta- 
tions at  instructor  activities  suggest  a  basic  dilemma:     Did  the  TICCIT  program 
relieve  Instructors  of  routine  duties  and  permit  them  to  make  better  use  of 
their  time,  or  did  It  generate  new  demands  that  prevented  any  freedom  of  choice 
Jn  allocating  their  time  to  activities?    Judging  from  the  attitudes  expressed 
by'  students  In  TICCIT  classes,  It  would  seem  that  instructors  p£»£  ^  °™ 
activities  in  English  courses  but  reacted  to  the  needs  of  the  situation 
in  mathematics  courses. 

Faculty  Acceptance.     A  survey  conducted  prior  to  the  Installation  of 
TICCIT  systems  at  the  community  colleges  Identified  the  major  dimensions  behind 
faculty  attitudes  toward  educational  practices  and  computer-assisted  Instruction, 
^clearest  and  most  prominent  dimension  behind  £ 'J™* 
educational  practices  was  a  focus  on  the  Interests  of  the  student.  Faculty 
seemea  to  base  their  responses  on  what  they  considered  best  for  students  and 
they  certainly  believed  'in  the  Importance  of  personal  interactions    n  student 
learning.     Other  dimensions  reflected  an  emphasis  on  the  subject  matter  of 
»  course    a  college's  receptivity  to  Innovation,  and  Incentives  for  student 
loZ    On  the  wio?:,  faculty  responses  to  statements,  about  educational  practices 
located  attitudes  supportive  of  the  premises  Inherent  to  the  design  o 
TICCIT  program.     In  contrast  to  the  four  factors  found  to  underlie  acuity 
beliefs  about  educational  practices,  two  factors  sufficed  for  describing 
facutty  attitudes  toward  computer-assisted  Instruction.     These  dimensions 
simply  reflected  general  attitudes  toward  CAI  and  familiarity  with  computers. 
Th!  majority  of  fLulty  members  lacked  any  prior  experience  with  computers  and 
thus  had  no  practical  basis  for  differentiating  among  the  statements  about 
computer-assisted  Instruction.     Most  often,  faculty  Indicated  that  they  were 
un^e  about  tne  probable  Impact  and  significance  of  -mputer-assl.ted  _ nstruc- 
tlon-     Certainly  there  was  an  openness  to  the  concepts  behind  the  TICCII 
Program  and  an  absence  of  any  apparent  school   resistance  to  cither  Innovative 
projects  or  Instructional  technology. 

Nearly  two  years  later  there  was  another  survey  of  faculty  attitudes  that 
explicitly  dealt  with  reactions  to  the  TICCIT  program-     The  five  dimensions 
found  on  this  survey  reflected  global  judgment,  of  ,he  TICCIT  program,  op  nions 
about  th.  management  of   Instruction  In  TICCIT  classes     Perception,  o     limit.  Ions 
in  the  TICCIT  program,  beliefs  about  the  teacher  s  role   In  TICCIT  (lasses,  una 
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familiarity  with  computers.    The  most  frequent  response  to  statements  callina 
for  a  global  judgment  of  the  TICCIT  program  was  'W  sure"  despUe  several  * 
academic  terms  of  experience  with  the  TICCIT  system.     Faculty  were  unsure  of 
whether  or  not  the  TICCIT  program  constituted  a  significant  development  in 
education  just  as  the^  had  earlier  been  unsure  about  CAI  in  general.     Th.J  had 

wou^f^^"  ?AI  W°Ul?  "ke  9tUdent8  actlve  in  <helr  ™  leading  and  S«  it 
would  tailor  Instruction  to  the  Individual  student;  the  TICCIT  pjogram^c 
these  expectations.     Prior  to  the  implementation  of  the  TICCtT  program  ^t  the 

re    ev:^J^uc\tLrnrallLbelieV?d  that  instruction  would 

relieve  instructors  of  routine  duties  and  enable  them  to  make  better  use  of 

'their  capabilities.     After  several  academic  terms  of  experience  with'the  TICCIT 
system,  faculty  expressed  uncertainty  that  the  TICCIT  program  haSaf fected 
instructor  duties  in  such  ways.     Instructors  had  become  llss  certain  Sat 

nr;amnf  instruction,  particularly  the  TICCIT  program,  would  benefit 
them  In  fulfilling  their  Instructional  responsibilities.    Moreover,  those 
instructors  closely  associated  with  the  TICCIT  project  often  reacted  to  the 
program  less  favorably  than  their  colleagues  in  otner  departments 


Implications 


Applications  of  the  TICCIT  Program.     The  TICCIT  program  had  a  slRnif leant 

fn  in  ch  ,!TCt  °"  Perfor—     This  held  undVmainline  applications' 

In  which  the  computer  became  the  primary  resource  for  student  learning.  Other 
applications  combining  regular  classroom  sessions  and  computer-assisted  instruc- 
ien«r8MWCd  C°Ur8e  C°°Pletlon  rate8»  "^ent  achievement  and  student  at tituaes 
?ICCI?  nroir7VSa;:  ^  ^  ln  ^tare-discussion  sections.  Tne 

TICCIT  program  had  been  intended  to  support  intact  courses  and  best  served  that 
purpose.     However,  instructors  played  a  critical  role  In  facilitating  sTudenc 
learning  on  the  TICCIT  system.     Students'  attitudes  toward  course's  d 
TICCIT  program  surpassed  those  of  students  ln  traditional  classes  onl"y  when 
instructors  elaborated  on  the  subject  matter  by  explaining  difficult  material 

JlCClH  ::ram1dn?dthe/rPUter  CU"1CU1-  with  —11  «ro«J  discussion^*1 

t  h  lessen  a  student's  need  for  the  assistance  of  an  instruc- 

tor, it  only  changed  the  nature  of  the  tasks  performed  by  teachers. 


a,.,.De8Pl     ^  lmProved  "udent  achievement  and  sometimes  favorable  student 
attitudes  made  possible  by  the  TICCIT  program,  there  were  dramatic  decreases  in 
course  completion  rates.    This  may  reflect  a  generic  problem  with  setf!p"ed 
ins  ruction  in  that  students  unable  to  manage  their  own  learning  fall  to 
satisfy  course  requirements.     Such  students  constitute  a  sizeable  percentage  of 
the  total  enrollment  at  community  colleges.     If  the  TICCIT  program'"" Inues 
-t  community  colleges,  there  should  be  careful  oritur Ing  „f  scudent  attendance 
and  exp  icit   Incentives  for  steady  course  progress.     Regular  contact 
Instructors  outside  of  sessions  on  the  TICCIT  system  and  small  group  discussions 
^mp^tion":^.^1^0"81  8tUd6nt  m°tlVatl0n  a"d  cour'sT8810"8 

The  results  of  this  evaluation  suggest  that  the  TICCIT  program  may  be 
Inappropriate  for  community  colleges.     Only  those  students  strong  In^Llr 
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initial  falllarlw  oith  the  aubject ' -tt.r  benefited  .ub.<:e«C^).|y  Irom 

Progr«"  ~e  ?ICCIT  piogra.  could  lead  to  unif orn,  po.itive  ~.ult,. 

a^Mn.  of  Co»r"-"-'"<"-H  Tn.tr.cti...    The  TICCIT  progra.  ha, 
confine  the  potential  ot  =~=g«5?32Si'iJS3^2S-t 
"ore."^-^ 

or^tices.  These \ncre~nt.  in  student  achievement  amounted^  ten  P«"ent 

?^;ts\rh^.r™ 

t.:or:h!:\ran  tnLT;£*:«denta^rUcture-di.cu..io„  claaaea.    This  denon- 
,tr."o„  provide,  further  evidence  for  advocating  school  adopt JJ^J^S 
assisted  instruction.    But  widespread  echool  adoption  of  co»»uter-sssletea 
^.tructionano  expectation,  of  significant  increase,  in  school  productivity 
would  be  premature. 

Computer-assisted  instruction  by  itself  guarantees  no  ma^aoluttons  to 
hv,»  oroblem,  faced  in  education.    Results  will  depend  on  the  instructional 
oesiJ  of  Se  curriculum  materials,  on  the  manner  of  implementation,  on  the 
naliS  of  the  student  population,  and  on  the  role  played  by  teachers.    It  is 
arable  tnat  computer  .applications  in  education  start  with  a  "cognition  of 
fchTff actors  involved  in  tne  instructional  process  and  seek  to  use  each  to 

'  S  ?i Lv:^ 

Providinffelnfack  on  student,  performance  and  its  potential  for  engaging 
students  as  ac^ve  participants  in  their  own  learning. 
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Chapter  1 

Introduction  ^ 

In  1971  the  National  Science  Foundation  awarded  a  contract  to  the  MITRE 
Corporation  to;  develop  and  demonstrate  the  TICCIT  program.    This  program  uses 
mini-computers  and  modified  television  receivers  as  the  heart  of  a  computer- 
assisted  instructional  system,  thus  its  acronym:     Time-shared,  Interactive, 
Computer-Controlled  Information  Television.    At  first  the  CAI, Laboratory  at 
the  University  of  Texa,s  and  later  the  Institute  for  Computer  Uses  in  Education 

i  ■ 

at  Brigham  Young  University  assumed  responsibility  for  the  design  and  production 
of  currits^r  materials  for  the  TICCIT  system.     The  TICCIT  project  began  with  ' 
the  combined  talents  of  engineers  and  educators.     It  followed,  therefore,  that 
the  system  join  available  .technology  components  and  innovative  teaching  stra- 
tegies together  so  as  to  form  a  complete  program- tailored  for  the  delivery  of 
instruction  and  the  promotion  of  student  learning  on  an  individual  basis. 

The  TICCIT  project  overcame  significant  obstacles  in  technical  and  instruc- 

f  "  • 

tional  design.    Now  it  puts  computer-assisted  instruction  for  entire  courses  in 
preealculus  mathematics  and  English  composition  within  the  financial  reach  of 
colleges,  at  least  in  terms  of  system  purchase  costs.     Whether  the  educational 
outcomes  probable  with  the  TICCIT  program  justify  such  a  purchase  is  a  difficult 
cost-benefit  issue  that  ultimately  reduces  to  subjective  judgments.  Here 
we  provide  information  on  the  educational  impact  of  the  TICCIT  program.  This 
first  chapter  reviews  the  general  conditions  which  led  to  the  project,  points 
to  several  key  features  that  attracted  attention  to  the  project,  and  then 
presents  a  description  of  the  TICCIT  system.     It  concludes  with  statements  of 
the  developers'  goals  and  fevaluator's  purposes. 
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1.1    Background  *C 

The  computer  industry  went  through  a  period  of  tremendous  growth  between 
1960  and  1970.    During  that  period  the  number  of  schools  with  computers  increased 
from  approximately  200  to  over  1,230  (Comstock,  (972a).    Expenditures  on  com- 
puters took  a  similar  jump.    Just  within  higher  education  expenditures  went  from 
$49  million  in  1962-63  to  $352  million  in  ,1968-69  (Comstock,  1972a)  .    Given  such 
phenomenal  growth  it  was  natural  to  view  computers  as  the  emerging  technology 
with  major  implications  for  educational  practice. 

Indeed  there  were  optimistic  predictions  for  a  revolution  within  education 
and  aggressive  efforts  by  the  computer  industry  to  capture  this  market  for  new 
equipment.    A  report  by  the  Carnegie  Commission  on  Higher  Education  (Kerr,  .1972) 
estimated  that  the  role  of-  instructional  technology  in  higher  education  would 
expand  through  research,  administration,  and  libraries  until  there  was  wide- 
spread use  for  instruction  after  the  year  2000.    Major  computer  firms  began  new 
divisions  devoted  to  instructional  systems.    Hardware  configurations  and  support 
packages  specifically  tailored,  for  education  appeared  on  the  market. 

Research  and  development  struggled  to  keep  pace  with  the  computer  explosion 
in  education.    Catalogs  listed  numerous  computer  programs  written  for  specific 
applications  (e.g.,  Lekrfn,  1971).    The  level  of  activity  and  interest  in  such 
computer  applications  prompted  the  formation  of  information  exchanges  to  spread 
the  word  about  available  programs  and  encourage  dissemination  (e.g.,  ENTELEK, 
1971).    There  was  a  parallel  increase  in  the  literature  about  computer  applica- 
tions for  instruction  (e.g.,  Razik,  1971),  most 'of  it  descriptive  or  exploratory 
in  nature* 

The  computer  as  a  resource  had  taken  hold  in  education.    Even  closer  to 
today  forecasts  still  indicate  a  growth  rate  above  15  percent  annually  for  our 
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expenditures  on  computer  hardware  (Rockart  &  Scott  Morton,  1975).    Again,  the 
magnitude  of  these  investments  would  suggest  a  substantial  impact  upon  the 
educational  process. 

Certainly  the  computer  has  become  an  invaluable  topi  in  educational 
research  and  school  administration.    Instruction  about  computers,  how  to  write 
programs  or  design  systems  for  example,  has  become  commonplace  in  our  schools 
and  collies.     But  direct  applications  of  the  computer  to  transmit  instruction, 
to  convey  information  to  students,  accounted  for  a  scant  amount  of  the  total 
computer  activity  in  schools  at  the  start  of  the  1970s  (Comstock,  1972a). 
This  was  true  despite  the  potential  of  the  computer  as  a  medium  for  meeting  the 
needs  of  individual  students  and  reshaping  the  learning  process  (see  Bushnell  & 
A.J.len,  1967;  Coulson,  1970). 

There  were  illustrations  of  successful  applications  of  the  computer  for 
instruction.    These  ranged  from  programs  in  arithmetic  for  elementary  schools 
(Suppes  &  Morningstar,  1969)  to  projects  in  physics  for  college  students 
(Kromhout,  Edwards,  &  Schwartz,  1969).     Rather  than  give  conclusive  evidence 
Sn  the  results  from  computer-assisted  instruction  (CAI)  and  thus  promote  wide- 
spread use,  such  exemplary  applications  seemed  to  strengthen  arguments  about 
the  ^tential  of  the  computer  as  an  instructional  resource.     Certain  studies  did 
suggest  significant  differences  in  favor  of  a  CAI  supplement  to  traditional 
instruction  (see  Feldhusen  &  Szabo,  1969; - Vinsonhaler  &  Bass,  1972  for  brief 
reviews).     Some  demonstrated  an  improvement  in  teaching  efficiency  since 
students  took  less  time  to  learn  a  given  amount,  of  material  (e.g.,  Bitzer  & 
Boudreaux,  1969;  Ford,  Slough,  &  Hurlock,  1972).     Other  comparisons  of  achieve- 
ment results  between  regular  classrooms  and  experimental  classes  supplemented  with 
CAI  showed  a  positive  effect  for  CAI  (e.g.,  Butler,  1969;  Culp  &  Castleberry, 
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1971).    Most  studies  reported,  in  anecdotal  or  descriptive  form,  favorable 
student  reactions  to  CAI  (see  King,  1975  for  a  review).    Still,  schools  had 
not  /uiop ted  CAI  on  a  grand  Scale.  -  •. 

A  variety  of  factors  might  be  cited  as  contributing  to  this  resistance,  or 
at  least  reluctance,  to  adopt  CAI.    Advocates  of  instructional  technology  tended 
to  list  a  few  major  problems:    inadequate  federal  resources  for  CAI  research  and 
development;  teacher  resistance  to  such  innovations;  lack  of  commitment  from  the 
commercial  sector;  and  the  need  for  a  majJr  demonstration  with  national  promi- 
nence as  an  impetus  toward  the  adoption  of  CAI  (e.g. ,  Anastasio  6,  Morgan,'  1972; 
A^sey  *  Dahl,  1973;  Holland  &  Hawkins,  1972).     Cost  must  be  considered  as 
another  major  factor  inhibiting  the  use  of  CAI  ( Judd, , 1971).     Perhaps  even  the 
absence  of  an  independent  evaluation  played  a  role  in  schools'  hesitation  to 
invest  in  instruction  with,  computers. 

The  rapid  growth  of  the  computer  as  an  educational  resource  and  schools- 
reluctance  to  adopt  CAI  may  seem  contradictory .  ' But  most  expenditures  on  com- 
puters were  confined  to  admMistrative  data  processing,  research  applications  or 
instruction  about  programming  and  hardware  design.     It  is  important  to  reiterate 
the  fact  that  only  a  small  proportion  of  the  dollars  spent  on  computers  went  to 
direct  instructional  applications  in  which  the  computer  served  as  the  mechanism 
that  did  the  teaching.    Despite  an  expanding  marl-*  for  the  computer  within 
education  and  a  growing  body  of  evidence  on  ^benefits  of  instruction  with 
computers,  schools  had  yet  to  adopt  CAI.     To  a  great  extent,  the  CAI  activity 
that  did  occur  was  limited  to  what  the  federal  government  supported. 

The  MITRE  Corporation  proposed  to  break  this  log  jam  in  the  flow  of  CAI 
to  schools.    Their  TICCIT  (Time-shared,  interactive,  Computer-Controlled 
information  Television)  system  was  to  be  a  small,  cost-effective  alternative 


ERIC 


r  31 


to  the  larger  CAI  systems  then  planned  (Stetton,  1971).    it  would/bring  t(* 
cost  of  CAI  down  to  an  affordable  level  for  most  schools  and  thus  encourage  the 
mass  dissemination  of  CAI  through  private  industry.    There  was  considerable 
support  for  the  demonstration  of  such  a  system  in  order  to  prove  the  merits  of 
computer-assisted  instruction  for  teaching  and  reduce  per  student  costs  Irf  a 
tight  economy. 

1,2    Key  Features  of  the  TICCIT  Project 

As  a  nonprofit  organization  MITRE  sought  and  received  a  federal  grant  to 
elaborate  on  the  design  of  the  TICCIT  system.  This  grant  followed  an  already 
substantial  investment  of  MITRE 's  own  funds  to  study  the  economic  feasibility 
of  CAI.  In  1971  MITRE,  in  conjunction  with  the  CAI  Laboratory  at  the  University 
of  Texas,  began  to  specify  the  design  elements  necessary  for  a  cost-effective 
application  of  CAI.  *  Their  initial  work  culminated  in  a  proposal  to  install  the 
TICCIT  system  in  schools  and  demonstrate  its  Impact  on  the  educational  process. 

There  were  several  unique  features  in  the  plans  for  the  TICCIT  project.  At 
least  five  points,  when  taken  together,  distinguished  the  TICCIT -program  f£m 
other  CAI  applications.     These  mighr  be  restated  here  as:     (1)  market  success; 
(2)  local  facility;   (3)  available  hardware;   (A)  leaner  control;  and  (5)  "main- 
line" instruction.     None  of  these  concepts  can  convey  much  about  TICCIT  without 
some  explanation  of  what  each  implies  about  the  rationale  for  the  TICCIT  project. 

Market  Success.     From  its  inception  the  TICCIT  program  held  a  market  success 
as  its  primary  goal  (see  MITRE,  1972).     It  was  an  explicit  attempt  to  create  a 
supply-demand  situation  conducive  to  a  commercial  success  for'CAI.     The  developers 
argued  that  low  cost,  high  quality,  or  technical  sophistication  would  not 
guarantee  school  adoption  of  CAI.     Of  course  these  would  be  necessary  conditions 
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for  a  market  success,  but  not  in  themselves  sufficient.    The  TICCIT  program  was 
-to  serve  as  a  catalyst  in  the  mass  dissemination  *  CM  through  its  systematic 
application  to  educational  needs.    Commercial  vendors  of  computers  and  publishers 
of  instructional  materials  had  forsaken  the  potential  market  for  CAI.    Major  computer 
firms  had  embarked  on  ambitious  marketing  programs  in  the  1960's  only  to  abandon 
instructional  applications  as  a  result  of  their  quite  limited  success.  TICCIT's 
developers  hoped  to  rekindle  their  interest  with  a  strong  example  of  a  market 
success.    Schools  in  search  of  a  CAI  system  would  provide  a  powerful  incentive 
for  private  industry  to  reenter  the  education  market. 

Local  Facility.    Just  as  market  success  represented  a  bold  goal,  MITRE  also 
took  a  distinctive  approach  to  the  economics  of  CAI.     The  TICCIT  system  was  to 
be  a  small,  local  facility  built  around  a  marriage  of  mini-computers  and  tele- 
vision technology.    It  would  be  small  enough  for  a  school  of  modest  size  (perhaps 
1200  students)  to  purchase  and  install  its  own  intact  system.    Such  a  local 
facility  provided  an  alternative  to  the  large-scale  system  for  CAI  then  under  *. 
development  (i.e.,  Bitzer  &  Skaperdas,  1971).    Arguments  about  the  relative 
'  advantages  of  small  and  large  computer  systems  continue  today  since  each  type 
has  its  strengths.    Here  it  is  sufficient  to  note  that  the  TICCIT  project 
stood  on  one  side  of  that  small-large  continuum  for  system  size. 

enable  Hardware.    Costs  had  to  be  kept  low  if  a  school  were  to  be  able 
to  afford  a  TICCIT  system.    MITRE  advocated  the  use  of  available  hardware  com- 
ponents in  order  to  reduce  CAI  costs  to  such  a  level.    The  TICCIT  system  . 
combined  computers  already  on  the  market  with  proven  television  technology. 
Its  uniqueness  stemmed  from  the  total  system  design  rather  than  the  character- 
istics •^*±&*  Part.    Indeed  its  components  could  be  purchased  "off-the- 
shelKld  then,  with  minor  modifications,  assembled  to  form,  the  basis  for  a  TICCIT 
system:  Yet  a  system  constructed  in  this  manner  could  display  a  rich  variety 
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of  Instructional  materials  as 'well  as  process  students'  responses  to  questions 
and  follow  their  directions  for  an  appropriate  teaching  strategy. 


Learner  Control .    The  capability  to  accept  students'  directions  for 
teaching  was  the  result  of  an  Innovative  approach  to  Instructional  design. 
Through  options  built  into  the  TICCIT  system  a  student  could  exercise  control 

s 

dver  his- own  instruction.    Each  student  could  choose  a  topic  for  study,  a  dif-  V 
ficulty  level  at  which  to  work,  and  an  instructional  sequence  for  his  or  her   *  j 
learning.    The  menu  of  possible  student  choices  defined  the  limits  of  learaer\  * 
control  over  difficulty  and  sequence,  and  the  structure  given  to  the  subject 
matter  restricted  the  range  of  topic  selections.    This  approach  to  learner  con- 
trol combined  our  knowledge  about  how  to  present  material  (Merrill  &  Boutwell, 
1973)  with  that  on  how  to  structure  content  (Gagne,  1970).    It  also  gave  ^ 
recognition  to  differences  among  students  and  attempted  to  meet  the  needs  of 
individual  students  through  flexibility  in  teaching  method. 

Mainline  Instruction.    The  concept  of  "mainline"  instruction  was  a  signi- 
f leant  departure  from  the  familiar  use  of  computers  as  a  supplement  to  class%> 
instruction.    Rather  than  serve  as  an  adjunctive  resource^r  th^  teacher,  the 
TICCIT  program  itself  was  to  be  the  primary  source  for  instruction.    It  would 
give  students  an  opportunity  to  study  at  separate  rates  of  progress  and  free 
teachers  for  concentrated  work  with  individual  students.    In  effect,  the  TICCIT 
program  would  also  make  it  possible  for  school  administrators  to  increase  the 
student- teacher  ratio  and  thus  reduce  the  labor-intensive  aspect  of  costs  in 
education. 

This  last  distinctive  feature  of  the  TICCIT  program  begins  to  touch  on 
the  manner  of  application  advocated  by  the  program's  developers.    While  other 
features  of  the  computer  system  certainly  contributed  to  its  potential  for 
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/♦along  with  the  key  features  of  the  program.    These  involve  MITRE' s  choice  of 
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the  community  college  as  an  appropriate  level  of  education  for  a  demonstration 
of  the  TICCIT  program,  and  their  decision  about  which  curricular  materials 
belonged  on  the  TICCIT  system. 

MITRE  sought  to  avoid  school  resistance  to  CAI  and  therefore  chose  community 
colleges  as  an  appropriate  test  case  for  market  success.    Community  colleges  had 
to  meet  the  diverse  needs  of  the  local  population  as  well  as  the  demands  of  in- 
creased enrollment  (see  Bushnell,  1973).    Their  commitment  to  Student  service 
tbgether  wi*h  growth  in„  enrollment  under  open  admissions  made  community  colleges 
receptive  to  such  changed  as  the  TICCIT  program.     In  addition  this  segment  of  higher 
education  seemed  to  be  lowest  in  the  proportion  of  schools  with  access  to  CAI 
(Comstock,  1972b). 

Just  as  the  choice  of  the  community  college  was  the  result  of  careful 
thought  about  a  potential  market,  so  the  selection  of  subject  matter  to  be  taught 
on  therTICCIT  system  hinged  on  economic  and  educational  considerations.  MITRE 
estimated  that  as  much  as  one-fifth  of  a  student's  program  of  study  took  place 
in  remedial  or  introductory  courses  offered  in  the  mathematics  and  English 
departments.    This  meant  that  a  small  number  of^courses  represented  a  major 
part  of  the  community  college's  curriculum  in  terms  of  student  enrollment.. 
Further,  there  was  a  pronounced  need  for  improved  instruction  in  algebra  and 
writing  skills.    A  large  percentage  of  students  do  not  complete  their  intro- 
ductory algebra  courses,  and  it  has  become  difficult  for  community  colleges 
to  deal  with  the  variety  and  extent  of  student  deficiencies  in  writing.  Mathe- 
matics and  English  were  selected  as  the  target  courses  for  the  demonstration  of 

the  TICCIT  program. 
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Its  developers  believed  that  the  TICCIT  program  had  the  potential  for  sub- 
stantial impact  on  instruction  in  community  colleges,  particularly  instruction 
*in  algebra  and  Writing,    In  order  to  demonstrate  this  impact  MITRE  proposed  the 
installation  of  TICCIT  systems  at  two  test  sites.    The  Alexandria  Campus  of 
the  Northern' Virginia  Community  College  Ind  Phoenix  College  of  the  Mariflbpa 
County  Community  College  District  agreed  to  serve  al  field  locations  for  the 
demonstration*  > 

1«3    Description  of  the  TICCIT  System 

The  concept  of  a  system  begins  to  describe,  the  products  as  well  as  the 
activities  of  the  TICCIT  project.    In  a  traditional  sense  a  computer  system  for 
instruction  has  hardware,  software  and  curricular  materials  or  courseware. 
The  TICCIT  program  also  had  other  subsystems.     Its  procedures  for  developing 
curricular  materials,  plans  for  installing  and  operating  computers  to  do  college 
teaching,  and  manuals  for  introducing  faculty  to  TICCIT  were  as  much  parts  of 
the  overall  system  as  the  computers  themselves.    Still,  an  elaboration  of  these 
activities  comes  later  (see  Chapter  3:     Implementation  of  the  TICCIT  Program) 
and  here  we  concentrate  on  the  familiar  components  of  a  CAI  system. 

The  TICCIT  instructional  system  as  described  below  is  the  same  as  that 
intended  for  use  in  community  colleges.    Other  configurations  exist  for  military 

training  and  special  education  (see  Rappaport  &  Olenbush,  1975)  but  these  were 

*  / 

V 

not  part  of  the  MITRE  proposal  for  a  field  demonstration  and  came  after  the 
Initial  work. for  that  demonstration  had  begun.    The  Information  presented  here^ 
is  taken  primarily  from  the  MITRE  Corporation's  own  description,  An  Overview  of 
the  TICCIT  Program  (1976).     Further  details  about  the  TICCIT  system^  especially 
those  components  designed  specifically  for  the  TICCIT  project,  can  be  found  in 
that  report. 
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1.3. a  '  Hardware 

"  The  hardware  for  the  TICCIT  system  is  what  a  visitor  to  a  site  would 
see:    physical  devices  that  look  like  grey  boxes  m  assorted  sizes  and  television 
sets  with  keyboards- similar  to  those*  on  electric  typewriters.    Actually  these 
are  the  computers  and  their  peripheral  equipment,  including  the  subsystems  built 
*4for  the  TICCIT  project  and  the  terminal  stations  at  which  students  learn.  Much 
of  the  equipment  is  available  for  purchase  commercially  such  as  the  two  Nova  800 
computers  from  Data  General  or  the  Century  114  disc  drives.    Other  pieces  of 
hardware  had  to  be  adapted  for  the  TICCIT  system,  for  example  the  color  tele- 
vision sets  that  function  under  computer  control.    A  third  category  of  equipment 
had  to  be  constructed  for  specific  needs  peculiar  to  the  TICCIT  system,  like  the 
devices  that  translate  digital  messages  from  the  computer  into  video  displays 
for  the  color  televisions. 

Figure  1.1  presents  a  schematic  diagram  of  the  TICCIT  computer  system.  The 
system  revolves  around  two  mini-computers  and  their  associated  devices  for  storage 
and  input-output.    This  basis  for  the  system  is  off-the-shelf  technology  that  is 
readily  available  for  purchase.    The  digicolor  system,  audio  system  and  graphic 
digitizer  represent  unique  hardware  elements  built  to  satisfy  specific  needs. 
The  terminals,  absent  from  the  diagram  but  certainly  an  essential  part  of  the 
whole  system,  constitute  the  most  visible  part  of  the  system  since  it  is  where 
the  student  interacts  with  the  computer. 

Computers  and  Peripherals .    The  TICCIT  system  has  a  dual  processor.  One 
mini-computer  serves  as  the  main  processor  and  another  as  the  terminal  pro-  "\£ 
cessor.    The  main  processor  has  a  larger  memory  (65,536  words  of  core  storage) 
and  does  the  bulk  of  the  work.    It  uses  the  courseware  data  base  to  assemble 
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textual  displays  and  process  students'  answers  to  problems  as  veil  as  their 
directions  for  teaching  strategy*    The  main  processor  also  maintains  student 
records  and  protects  both  student  records  and  system  memory  against  power 
failures  or  unforeseen  system  crashes. 

In  order  for  the  main  processor  to  fulfill  these  responsibilities  the  ter- 
minal processor  has  to  handle  the  input-output  demands  of  128  terminals*  The 
terminal  processor  r^eives  keyboard  entries  from  the  terminals  and  transfers 
complete  messages  over  to  the  main  processgr.     It  then  feeds  back  textual  dis- 
plays or  generates  audio  messages  and  graphic  material  as  directed  by  the  main 
processor.    The  terminal  processor  does  little  of  the  actual  computation  like 
judging  answers,  determining  sequence,  or  updating  lrecords.     Instead  it  Juggles 
input-output  messages  for  128  terminals  and  services  them  so  that  it  seems  as  if 
each  terminal  has  a  dedicated  communication  link  with  the  computers  and  no  other 
terminal  competes  for  time.  ^ 

The  discs  that  go  along  with  the  processors  provide  storage  for  data  that 
correspond  to  the  processors'  tasks.     Since  the  main  processor  forms  frames 
of  text  and  keeps  records  of  students'  progress,  it  has  access  to  disc  storage 

for  courseware  data  as  well  as  student  data.    Two  other  moving  head  discs  for 

r 

the  courseware  data  necessary  in  generating  graphics  and  constructing  audio 

# 

messages  fall  under  the  control  of  the  terminal  processor.  Each  of  these  five 
discs  has  a  memory  capacity  of  12,000,000  words  in  contrast  to  the  processors' 
core  storage  of  less  than  70,000  words.  There  are  also  two  smaller  fixed  head 
discs:  one  which  stores  a  system  directory  and  another  which  buffers  keyboard 
data.  The  directory  gives  the  location  of  data  within  the  system  and  lists 
valid  user  identification  numbers.     Such  a  memory  bank  is  essential  for  a  CAI 
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systea  based  on  mini-computers.    With  this'conf igurstion  the  TICCIT  system 
holds  the  textual  materials  necessary  for  five  full  semester  courses  on 
three  moving  head  discs.     Two  additional  discs  store  the  data  bases  for 
audio  and  graphics  messages. 

Besides  disc  drives  the  TICCIT  system  includes  a  somewhat  standard  array 
of  peripheral  equipment.    As  shown  on  the  left  side  of  the  diagram  for  the 
TICCIT  computer  system  (Figure  1.1),  there  is  a  card  reader,  a  magnetic  tape 
unit,  a  line  printer,  and  a  console  for  system  operation.    These  peripherals 
facilitate  courseware  development  and  help  fulfill  the  administrative  functions 
that  accompany  a  mainline  CAI  system. 

Special  Devices.    As  the  first  instructional  system  to  combine  computer 
and  television  technology,  the  TICCIT  program  faced  some  unique  problems  that 
required  special  hardware  subsystems.     MITRE  designed  a  digicolor  system  in 
order  to  convert  digital  computer  messages  to  display  signals  consistent  with 
television  receivers.    The  display  generator  decodes  the  computer  message  and 
transfers  a  single  frame  TV  picture  over  to  the  video  refresh  memory.  This 
refresh  memory  then  sends  that  single  picture  to  its  associated  TV^recelver 
and*  continues  to  repeat  the  picture  so  that  it  appears  as  a  steady  display. 
The  refresh  memory  has  a  solid-state  design  with  one  printed  circuit  board  for 
luminance  and  another  for  color.     Of  course  TV  receivers  also  accept  full 
color  videotape  signals:     the  TICCIT  system  has  a  set  of  20  video  cassette 
tape  players  which  the  computer  directs,  although  an  operator  must  perform  the  f 
manual  operations  like  loading  a  given  tape.     Since  the  digicolor  system  sends 
television  pictures  over  coaxial  cables,  the  TICCIT  signals  can  be  compatible 
with  normal  cable  TV  reception. 
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The  TICCIT  audio  system  holds  about  2  1/2  hours  of  sentences  and  phrases 
recorded  as  digital  information.    Each  second  of  these  hours  is  stored  as 
20,000  bits,  thus  the  need  for  a  large  disc  memory  to  serve  as  the  audio 
data  base.    Attached  to  the  terminal  processor  is  a  digital  to  analog  converter 
that  retrieves  audio  messages  from  disc  storage  via  a  shift  register  buffer, 
and  then  passes  the  messages  through  a  switching  System  to  the  TV  receivers. 
Another  converter  handles  the  reverse  process  in  courseware  development:  it 
accepts  a  continuous  voice  signal  and  stores  it  as  binary  information.  The 
complete  audio  system  can  transmit  simultaneous  messages  to  approximately 
twenty  receivers. 

A  third  special  subsystem  permits  the  semiautomatic  entry  of  graphics 

into  the  courseware  data  base.    This  is  an  optional  system  component  as  it 

» 

facilitates  courseware  production  rather  than  delivery.    The  actual^roduction 
of  graphic  materials  still  takes  place  outside  of  the  TICCIT  system,  but  there 
is  a  graphic  digitizer  that  scans  artists'  work  and  records  it  as  part  of  the 
courseware  data  base  connected  to  the  terminal  processor.     In  the  graphics 
digitizer  a  modified  TV  camera  scans  an  artist's  drawing  and  records  the 
luminance  and  color  data  for  the  graphics  on  r.hat  paper.    There  is  an  edit 
program  for  the  graphics  source  file  so  an  artist  can  later  check  the  results 
of  the  camera's  scan  and  make  changes  to  the  actual  display  frame  as  necessary. 
With  the  projection  for  over  10,000  pieces  of  graphic  material  in  the  courses 
under  development,  the  partial  automation  permitted  by  the^graphics  digitizer 
would  contribute  to  a  smooth  production  process. 

Terminals.     The  TICCIT  terminal  is  really'  a  color  television  set  and  an 
electronic  keyboard.    As  modified  for  the  TICCIT  system  the  television  sets 
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display  alphanumeric  information  along  17  lines  with  43  character  positions  in 
a  line.     Characters  themselves  fall  under  computer  control;  programs  can  define 
as  many  as  512  distinct  characters  for  use  at  one  time.    The  dimensions  for  a 
graphic  display  would  be  204  elements  in  the  vertical  direction  by  430  horizontal 
elements.     Both  alphanumeric  and  graphic  displays  take  advantage  of  the  color 
capabilities  of  the  T1CCIT  system  to  highlight  key  words,  provide  attractive 
material,  and  otherwise  cue  student  attention.    While  such  single* frame  displays 
use  seven  colors,  the  terminals  can  also  receive  full  color  videotapes  with  sound. 
These  display  capabilities  together  with  access  to  audio  messages  give  the 
TICCIT  system  a  rich  variety  of  instructional  materials,  especially  in  contrast 
to  the  teletypes  and  CRT  consoles  available  at  the  start  of  the  TICCIT  project. 
Further,  the  use  of  a  color  television  as  a  CAI  terminal  holds  promise  for  other 
applications  in  the  home  based  on  coaxial  cable  communications. 

The  keyboard  through  which  the  student  directs  his  instruction  and  responds 
to  the  TICCIT  system  resembles  the  key  set  on  a  standard  typewriter  (see  Figure 
1.2).   -That  standard  set  of  characters  is  sufficient  for  textual,  numerical,  and 
symbolic  entries.     A  block  of  keys  on  the  left  side  of  the  keyboard  provides  the 

r 

student  with  some  common  controls  for  editing  entries  and  positioning  the  cursor 
on  the  screen.    More  important  to  the  design  of  the  TICCIT  program  is  the  block 
of  keys  on  the  right  side  of  the  keyboard  that  defines  a  student's  choices  in 
TICCIT's  teaching  strategy.    This  learner  control  is  an  integral  part  of  the 
entire  system  as  it  at  once  defines  the  extent  of  flexibility  in  teaching 
strategy  and  restricts  the  instructional  design  compatible  with  the  TICCIT  system. 
As  installed  at  the  demonstration  sites  the  TICCIT  program  supports  instruction 
under  a  learner  control  paradigm  that  is  consistent  across  lessons,  courses, 
subject  fields,  and  schools. 
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1.3.b  % Software 

Software  is  what  enables  the  equipment  in  a  T1CCIT  computer  system 
to  fulfill  specific  tasks.     For  the  main  processor  to  shuffle  student  data 
from  disc  to  computer  memory  and  backt  or  the  terminal  processor  to  coordinate 
the  delivery  of  textual,  graphic,  and  audio  messages  to  over  100  terminals,  or 
the  computer  system  to  judge  a  student's  response  to  a  question  as  right  or 
wrong  requires  software.    Without  such  programs  there  would  be  no1  flashing- 
lights,  spinning  tapes,  or  students  learning  on  the  TICCIT  system* "">l£ 

The  computer  programs  written  for  the  TICCIT  system  fall  into  .rare  func- 
tional categories.     Operating  systems  and  application  programs  relate  to  the 
delivery  of  instructional  materials.    Two  other  program  packages,  namely  data 
entry  and  courseware  processing,  support  the  production  and  assembly  of 
materials.  A  fifth  function  of  the  software  is  to  provide  utility  services 
such  as  system  protection  against  power  fluctuations  and  disc  management. 

Operating  Systems  and  Application  Software.     Together  the  operating 
systems  and  application  programs  allow  mini-computers  to  deliver  courseware 
and  follow  TICCIT's  learner  control  paradigm  for  128  terminals.     There  is  an 
operating  system  for  the  main  processor  that  brings  courseware  programs  into* 
core  and  executes  them  as  appropriate  for  each  student's  instruction.    A  time 
limit  on  execution  (79  milliseconds  for  a  full  load  of  128  terminals)  prevents 
one  student's  needs  from  interfering  with  instruction  for  other  students,  yet 
^ives  adequate  time  for  system  action.     The  main  processor  operating  system 
(MPOS)  also  keeps  an  accurate  data  record  for  each  student  and  feeds  this  to 
disc  for  use  with  the  next  cycle  of  courseware  execution. 
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The  terminal  processor  operating  system  (TPOS)  serves  as  an  interface/ 
between  the  main  processor  and  the  terminals.     It  actually  sends  messages 
to  the  terminals  under  MPOS  direction  and  passes  complete  student  entries  on 
to  the  main  processor  for  action.    The  routine  input-output  work,  such  as 
sending  a  student's  key  stroke  back  for  display  on  his  terminal  or  changing  an 
entry  as  a  student  edits  it,  comes  under  the  lisr-t>f  TPOS  tasks  as  do  a  number 

of  other  housekeeping  chores.    This  operating  system,  however,  is  far  from 

■  Lj  * 
routine  in  that  it  succeeds  irt  reliable  mini-computer  service  to  128  terminals. 

The  application  software  is  a  direct  result  of  the  approach  to  instruc- 
tional dtkign  taken  in  the  T1CC1T  program.     All  student  entries  rely  on  the 
application  software  for  analysis  and  subsequent  directions  for  a  system 
response.     Because  these  programs  must  be  in  constant  use,  each  resides  in 
core  as  an  assembly  language  program  in  order  to  promote  an  efficient  system 
response  to  student  entries.     It  is  with^his  collection  of  programs  that 
data  about  a  student's  status  begin.    The  application  software  sets  the  indica- 
tors and  records  the  variables  which  reflect  where  a  student  currently  stands 
in  the  courseware  as  well  as  the  quality  of  his  prior  work.    This  information 
is  essential  for  the  system  to  determine  an  appropriate  response  to  a  learner 
control  command  or  to  make  a  recommendation  about  how  the  student  should 
proceed. 

Data  Entry  and  Courseware  Processing.    An  entire  college  course  on  the 
TICCIT  system  occupies  a  large  number  of  display  frames,  just  as  a  textbook 
with  equivalent  material  would  run  to  thousands  of  pages.    There  is  software 
that  partially  automates  the  construction  of  these  materials.     For  graphics 
a  camera  scans  art  work  and  computer  programs  record  the  result  in  source 
files.     For  text  a  member  of  the  courseware  production  team  prepares  displays 
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for  rulpa,  examples,  and  practice  problems  as  well  as  the  specifications  for 

sequencing  these  frames  and  judging  students'  answers  to  problems.    Such  frame 

i         -  ■  • 

*  construction  takes  place  at  a  terminal  under  author  mode:     software  for  data 

emtcy  stores  these  frames  in  source  files  for  text.    Courseware  processing 

involves  the  assembly  of  these  source  files  and  other  data  (e.g.,  tests  and 

maps  of  coursewprfe  contents)  into  a  form  ready  for  delivery  to  students  as 

instruction.  ?* 

1.3.C  Courseware 

It  is  important  to  acknowledge  the  TICCIT  program  as  a  form  of 
mainline  instruction.     The  courseware  for  the  program  rests  on  the  assumption 
that  students  would  learn  all  the  material  required  for  college  credit  on  the 
TICCIT  system.     This  reverses  the  usual  relationship  between  the  teacher 
ax\d  the  computer  found  in  an  adjunctive  application  of  computer-assisted 
instruction.    Rather  than  supplement  classroom  work,  TICCIT  takes  primary 
responsibility  for  students'  instruction  while  the  teacher  becomes  an  additional 
resource.     So  "the  courseware  on  the  TICCIT  system  must  be  self-sufficient.  It 
must  contain  all  of  the* course  material  called  for  in  a  college's  curriculum 
standards  and  deliver  it  in  such  a  manner  that  students  learn. 

Obviously  mariniine  instruction  places  a  burden  on  the  TICCIT  program  with- 
out  precedent  from  other  CAI  systems.     Yet  the  replacement  of  traditional 
classroom  activities  is  a  forceful  argument  for  reduced  costs  through  CAI.  If 
the  TICCIT  program  under  an  increased  student-teacher  ratio  could  release 
teachers  from  classroom  lectures  and  free  them  for  other  tasks,  the  change  in 
teacher  productivity  might  justify  the  program's  expense.     The  developers  of 
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the  TICCIT  coursewares  initially  the  CAI  Laboratory  of  the  University  of  Texas 
and  later  the  Institute  for  Computer  Ufees  in  Education  at  Br igham  Young  Univer- 
sity, accepted  this  challenge  and  embarked  on  an  approach  to  courseware  produc- 
tion consistent  with  their  concepts  o£  market  success  and  mainline  instruction. 

Content,    After  a  study  of  potential  markets  for  CAI,  MITRE  chose  community 
colleges  as  the  segment  of  education  likely  to  be  most  receptive  to  the  TICCIT 
program.    Here  there  was  a  pronounced  need  for  basic  instruction  coupled  with 
a  willingness  to  experiment  with  new  methods  that  promised  an  improvement  in 
student  learning.     Indeed  enrollment  growth  at  community  colleges  had  begun 
to  strain  the  limits  of  faculty  teaching  loads  so  the  TICCIT  program  did  nO$fr 
present  a  threat  to  jobs.     Instead  it  offered  a  vehicle  to  meet  local  demands 
for  instruction. 

Within  community  colleges  certain  courses  represent  an  inordinate  share 
of  the  total  curriculum  in  terms  of  enrollment.    This  is  especially  true  for 
courses  in  algebra  and  writing  skills  where  the  numbers  of  students  at  a 
college  require  tens  of  sections  each  term.    Further,  if  each  student  took 
just  one  algebra  or  writing  course  per  term  over  a  full  two-year  program,  such 
courses  would  constitute  twenty  percent  of  the  college  curriculum  by  enrollment. 
MITRE  therefore  decided  to  implement  the  TICCIT  program  for  precalculus  mathe- 
matics and  English  composition.     Outlines  of  the  content  coverage  available  in 
the  mathematics  and  English  courseware  appeat  in  Appendices  A  and  B. 

Basically  the  mathematics  courseware  contains  the*  equivalents  of  introduc- 
tory algebra,  intermediate  alegebra,  and  elementary  functions.    Beginning  with  a 
review  of  arithmetic  skills,  the  algebra  tpaterials  span  common  logarithms, 
linear  equations,  probability,  arithmetic  and  geometric  progressions,  as  well  as 
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»  other  traditional  topics  like  polynomial  and  algebraic  expressions.  The 
coverage  for  elementary  functions  includes  polynomial,  trigonometric, 
e^onentlal  and  logarithmic  functions  with  a  brief  treatment  of  conic  sections 

* 

ill;  coordinate  geometry. 

v  ». 

.     »The  courseware  for  English  compositon  provides  the  equivalent  of  a  writing 
la|>o|atory  with  instruction  in,  grammar,  mechanics,  diction,  sentence  structure 
an|  paragraph  development.    Unlike  the  TICCIT  program  for  math,  much  of  the 
students'  work  takes  place  independent  of  the  T1CC1T  system  as  writing  assign- 
ments.   This  reflects  the  general  courseware  objective  for  composition:  clear 
and  effective  writing  in  a  standard  English  style. 

Structure.    When  a  student  begins  a  course  on  the  T1CC1T  system,  he  or  she 
encounters  a  series  of  maps.    These  maps  chart  the  subject  matter  to  be  learned. 
Just  as  a  textbook  has  chapters  and  sections,  a  T1CC1T  course  has  units  and 
lessons  presented  on  maps.     Figure  1.3  is  a  unit  level  map  for  the  precalculus 
courseware;  Figure  1.4  is  for  the  English  composition  courseware.  These 
figures  present  all  TICC1T  courseware:     a  single  math  course  might  contain  only 
four  such  units,  as  in  the  beginning  algebra  course  at  one  demonstration  site 
that  requires  units  23,  22,  20  and  17,  or  a  student  in  a  composition  course 
might  need  work  only  in  units  11,  10,  9  and  1  of  the  English  courseware.  Once 
a  student  chooses  a  unit  within  a  course,  there  is  a  map  of  the  lessons  within 
that  unit.    The  series  of  maps  continues  for  three  steps  from  units  to  lessons 

to  segments.    Each  step  yields  further  detail  about  the  content  coverage 

■         '  \ 
until  a  student  reaches  the  segments  within  a  lesson.    At  the  segment  level 

irtstruction.^oc'curs . 

-  Unlike  the  outline  fro r mat  of  a  table* of  contents,  the  maps  in  T1CCIT 

, ;  ■ 

courseware  adhere  to  a  hierarchical  structure.    A  segment  below  another  on 
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requisite  to  that  higher  tf^rafet  /  ,  fri  general  such  a  vertical 
connection  implies  that  learning  the  material,  found  lower  in  the  path  will 
facilitate  learning  in  later,  higher  positions.    For  example,  in  the  English 
courseware,  the  unit  on  writing  paragraphs  comes  before  that  on  organizing 
essays.    Units  adjacent  to  one  another  in  a  horizdntal  direction,  on  the  other 
hand,  share  no  definite  relationship.    Thus,  the  map  structure  for  the  subject 
matter  depicts  interrelationships  among,  first,  units  at  the  course  level, 
then  lessons.,  and  finally  Segments.  .  This  structures  student  learning  too  as 

prerequisites  determine^ content  sequence  and  adjacent  map  positions  provide  a 

S 

choice. 

There  is  another  dimension  of  structure  which  students  do  not  encounter. 
It  is  that  of  the  files  on  the  TICCIT  system.    Files  on  discs  hold  the  actual, 
instructional  materials.    This  approach  allows  the  storage  of  extensive  rule 
statements  and  instances  to  accompany  each  rule.    The  rule  states  the  concept 


to  be  learned  while  the  instances  provide  exan^les  of  ,its  application  and 
problems  for  student  practice.    Alternative  versions7^ of  a  single  rule  and  its 
corresponding  instances  constitute  a  segment  of  instruction.    All -courseware 
on  the  TICCIT  system  eventually  comes  down  to  the  level  of  segments,  and,  in 
turn,  files  of  rules  apd  instances. 

Strategy.    Much  of  the  TICCIT  strategy  for  teaching  arises  from  its  map 
structure.     Boxes,  whether  for  units,  lessons  or  segments,  colored  green  on  a 
map  display  indicate  that  a  student  has  mastered  that  material  as  demonstrated 
by  performance  on  tests  embedded  at  the  lesson  level  or  his  solution  of  practice 
problems  at  the  segment  level.    A  red  box  signifies  failure  on  a  le$son  test 
or  repeated  difficulty  with  practice  problems,  and  hence  a  need  for  further 
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work  on  this  material.    Yellow  represents  an  intermediate  state:    there  has 
been  work  on  the  material  but  not  enough  for  a  system  decision  about  student 
mastery.    When  all  the  units  in  a  course  map  turn  green,  the  student  has 
completed  that  course.    The  pretest  for  the  English  courseware  assigns  color 
codes  to  units  based  on  a  student's  results  for*  specific  objectives,  and  thus 
expedites  course  progress.    A  student  always  has  the  option  to  take  a  lesson 
test  first  and,  if  he/or  she  passes,  avoid  the  study  of  familiar  material. 

V  Besides  maps  the  TICCIT  strategy  for  teaching  relies  heavily  on  learner, 
control.    A  set  of  commands  available  at  the  segment  level  shifts  responsi- 
bility from  a  system  prescription  for  learning  to  a  student' s  .own  initiative 
in  his  learning.     Three  commands  correspond  to  statements  of  the  rule ' that 
students  must  master  (RULE),  examples  of  its  application  (EXAMPLE),  and 
problems  for  practice  on  its  application  (PRACTICE).     The  rule  and  instance 
files  limit  the  total  number  of  displays  for  these  commands;  but  the  student 
really  exercises  control  over  what  order  and  how  often  he  wishes  to  use  RULE, 
EXAMPLE ,  and  PRACTICE.     It  is  also  the  student's  prerogative  to  choose  a 
difficulty  level  for  examples  and  problems  through  the  EASY  and  HARD  commands. 
The  same  two  commands,  in  conjunction  with  a  RULE  display,  move's  student  to 
a  more  concrete  or  abstract  form  of  what  it  is  that  he.  or  she  must  learn. 

When  a  student  has  trouble  understanding  a  rule,  following  an  example, 
or  solving  a  problem,  that  student  seeks  further  clarification  through  the 
HELP  command.     Then  another  rule  display  attempts  to  lead  the  student  through 
a  simple,  step-by-istep  presentation.     Or  the  student  receives  assistance  on 
examples  and  problems  as  additional  detai Is  "isolate  the  attributes  of  the 
instance  relevant  to  solution  (for  example,  as  different  colors  or  format 
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poeitions  on  his  or  her  display)  while  textual  or  audio  messages  point  to  their 
importance.    When  the  TICCIT  system  detects  a  student  strength  or  weakness  on 
tests  and  exercises,  it  volunteers  advice  about  how  the  student  should  proceed'. 
A  student  can  request  such  recommendations  for  his  or  her  learning  strategy 
through  the  ADVICE  command.  V 

,A  learner  in  the  TICCIT  program  has  control,  but  not  complete  control,  over 
instruction.    Maps  begin  to  define  an  appropriate  sequence  in  terms,  of  the  sub- 
ject mstter.    The  set  of  available  commands  restricts  the  learner's  choices  at 
the  segment  level.    The  TICCIT  program  does  not  represent  learner  control  in  a 
natural  language,  tutorial  setting.     But  it  does  provide  learner  control  over 
a  sophisticated  drill  and  practice  environment,  and  its  instruction  draws  on 
recent  advances  in  defining  the  conditions  for  learning. 

1.4    Goals  of  the  TICCIT  Project 

As  with  other  innovations  in  education  the  TICCIT  project  began  with 
ambitious  goals.    Unlike  other  major  programs,  however,  the  TICCIT  developers  . 
set  forth  a  clear  statement  of  their  position  in  early  papers  and  held  to  the 
same  goals  throughout  the  project's  history.    This  simplified  the  task  of 
evaluation  and,  we  believe,  contributed  to  the  evaluator's  success  in  posing 
questiona,  collecting  data,  and  interpreting  results  pertinent  to  the  TICCIT 
program.     Here  we  present  the  original  Boals  of  the  TICCIT  project  from  the 
perspective  of  its  strongest  advocates— its  developers. 

This  project  involves  three  distinct   levels  ot  goals.     An  obvious  level 
within  the  field  of  Instructional  technology  is  the  creation  of  a  reliable, 
operational  system  with  certain  performance  characteristics.     Indeed  a  computer 
•cientist  might  consider  measures  of  system  parameters  like  lag' time  to  system 
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response  under  different  loads  sufficient  in  themselves  as  an  evaluation.  Based 
on  their  simulation  of  system  perfqrmance  MITRE  estimated  that  the  TICCIT  pro- 
gram  would:    support  over  100  active  terminals;  respond  to  completed  student 
entries  within  1/2  second;  echo  key  strokes  within  1/4  second;  execute  49,000 
courseware  instructions  per  student  input.    These  estimates  simplify  to  a  system 
easily  able  to  accommodate  student  instruction  on  an  Interactive  basis. 

Another  level  of  goals  relates  to  a  market  success.    Community  colleges, 
and  later  other  sectors  of  education,  would  adopt  CAI  as  an  important  new 
instructional  resource.    Private  industry  would  provide  capital  investment  for 
the  production  of  hardware,  software,  and  courseware,  as  well  as  invest  in  its 
dissemination  through  new  marketing' methods.    Government  and  foundations  would 
.support  major  programs  leading  to  the  application  of  CAI  to  meet  pronounced 
educational  needs.    Actually,  the  TICCIT  project  did  not  pretend  that  it  alone 
would  bring  about  such  long-range  goals.    Rather  it  was  to  serve  as  a  catalyst  * 
in  the  application  of  resources  for  developing  and  disseminating  CAI  for 
individualized  instruction. 

A  final  category  of  goals  deals  with  the  more  immediate  consequences 
of  the  TICCIT  project.    These  were  to  occur  as  a  result  of  the  demonstration 
at  community  colleges.    While  an  engineer  might  view  consequences  like  student 
learning,  as  remote  in  contrast  td  system  performance,  the  learning  that  takes 
place  on  TICCIT  constitutes  the  most  important  criterion  among  educators. 
It  was  the  instructional  psychologists  responsible  for  courseware  design  who 
first  elaborated  the  goals  of  the  TICCIT  program  with  regard  to  learning  (see 
Junderson,  1973).    Their  couseware  design  goals  covered  costs,  student  learning, 

t 

teacher  roles,  and  curriculum  content.    We  repeat  them  here  not  as  standards 
for  evaluation  but  as  initial  ambitions  of  the  TICCIT  program. 
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Cost  goals: 

—  less  than  $1.00  per  student  contact  hour 

This  figure  reflects  prorated  system  purchase  and  courseware  costs, *gs 
well  as  maintenance  and  support  personnel. 

—  25%  less  time  tp  course  completion 

It  would  take  students  this  much  less* time  to  complete  a  course  as 
compared  with  jfche  length  of  time  required  to  complete  equivalent  work 
in  traditional  classes. 

—  increase  ^enrollment  significantly 

As  a  result  of  lower  attrition  rates,  enrollment  in  advanced  courses 
wou],d  increase  while  the  popularity  of  TICCIT  would  attract  more  Students 
to  introductory  courses. 

Goals  for  students: 

—  85X  of  students  will  achieve  mastery  u 
Tests  embedded  in  the  courseware  at  the  lesson  level  define  riastery^the 
'percentage  refers  to  total  course  enrollment. 


—  gr«*ter  than  a  50%  Improvement  in  the  efficiency  of  student  work 
Between  the  start  and  conclusion  of  a  TICCIT  course,  students  would 
adapt  to  the  system  and  become  more  proficient  in  their  use  of  study 
time  on  the  computer. 

—  improvement  in  learning  strategies 

An  Improvement  in  strategy  would  accompany  that  in  efficiency  as 
evidenced  by  a  decrease  in  students'  reliance  on  TICCIT  advice  for 
guidance. 

—  voluntary  approach 

Students'  scores  on  attitude  measures  and  their  pursuit  of  optional 
course  material  would  indicate  a  positive  approach  to  the  subject  matter 
and  favorable  reactions  to  their  learning  experiences  with  the  TICCIT 
program.  s; 

—  responsibility 

With  independent  study  and  learner  control,  students  would  develop  a 
sense  of  responsibility  for  their  own  instruction;  this  would  be 
reflected  in  a  high  percentage  of  attendance  at  scheduled  times  for 
TICCIT  classes. 

Goals  for  educators:  ,  . 

^-  define  new  professional  roles  in  development  as  well  as  in  counseling- 

management  ,       .  . 

While  the  TICCIT  program  would  supplant  the  traditional  role  of  a  teacher 
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ln  the  classroom,  it  would  create  new  demands  for  contributions  to 
further  courseware  production  and  for  intensive  work  with  students 
on  an  individual  basis. 

—  stimulate  teachers  to  demonstrate  humane  values  in  follow-on  orv 
coordinate  instruction 

Content  goal: 

—  small  step  forward  in  content 

The  TICCIT  program  would  not  be  a  major  curricular  innovation:    it  would 
combine  beginning  and  intermediate  algebra  into  one  modular  system  and 
attempt  to  implement  a  single  course  to  replace  the  usual  sequence  of  ' 
algebra,  trigonometry,  and  analytic  geometry;  English  courseware  would 
adopt  a  generative  approach  to  composition. 

—Clarify  objectives  and  provide  flexibility 

The  specificity  necessary  in  courseware  production  would  force  clarity 
on  statements  of  content  objectives  while  the  modular  design  of  course- 
ware would  provide  flexibility  in  integrating  the  TICCIT  program  into  ' 
existing  curriculum  structures. 

Perhaps  only  an  evaluator  would  devote' serious  attention  to  such  goal 
statements.    Goals,  after  all,  tend  always  to  exceed  our  grasp.    But  funding 
agencies  consider  program  goals  in  allocating  resources,  schools  in  deciding 
about  participation  in  experimental.projects,  evaluators  in  making  Judgments 
about  data  needs,  and  others  in  forming  expectations.    Since  goals  influence 
decisions,  it  seems  reasonable  that  such  statements  balance  an  enthusiastic 
optimism  about  potential  accomplishments  and  an  historic^  realism  about 
results  of  prior  work.    Here  we  took  the  developer's  goajj,  as  earnest  state- 
ments of  intent  which  should  guide,  though  not  dictate,  our  attention  in  " 
evaluating  the  TICCIT  program. 

1.5    Purposes  of  This  Evaluation 

What  is  a  goal  for  a  developer  is  often  a  question  for  an  evaluator.  This 
evaluation  of  eaucational  impact  covers  the  breadth  of  TICCIT' s  potential  and 
goes  into  detail  on  key  issues.    The  questions  which  the  evaluation  addresses, 
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bower,  reflect  not  just  the  program's  goals  but  also  needs  for  information 
for  making  future  decisions  about  the  TICC1T  prpgram  and  similar  projects.  The 
community  colleges  involved  ib  the  demonstration  must  plan  how  best  to  use  the 
TICCIT  system  once  other  participants  complete  their  roles;  other  schools  want 
sufficient  data  to  decide  about  adopting  CAI  as  an  instructional  resource. 
Certainly  funding  agencies  must  set  priorities  in  allocating  their  resources 
and  professionals  in  fields  pertinent  to  instructional  technology  look  for 
promise  in  new  directions  for  research  and  development.    This  evalu- 
ation by  itself  does  not  satisfy  all  such  needs .    Rather  it  contributes  to 
the  information  available  to  diverse  audiences  in  making  these  decisions. 

The  evaluation  concentrated  on  documenting  outcomes  and,  to  some  extent, 
activities  in  four  basic  areas  of  inquiry.    These  were:    student  performance; 
faculty  acceptance  and  teachers'  roles;  implementation;  and  instructional 
conditions.    The  major  questions  posed  in  the  course  of  the  evaluation  follow. 

Student  Performance 

o  Student  achievement:     How  does  student  achievement  under  the  TICCIT 

program  compare  with  that  of  students  enrolled  in  regular  class  sections? 

o  Student  attitudes:    How  do  students  react  to  TICCIT  as  an  instructional 
program  in  contrast  with  traditional  practices? 

o  Student  activities:     On  what  do  students  in  TICCIT  classes  spend  their 
time?    How  does  this  differ  from  normal  class  activities? 

o  Student  conduct:    How  do  institutional  variables,  such  as  enrollment 
and  attrition,  reflect  student  performance  on  the  TICCIT  system? 

o  Interrelationships  in  student  performance:    What  characteristics  dis- 
tinguish among  TICCIT  students  according  to  criteria  of  achievement, 
course  completion,  and  total  time  spent  on  the  TICCIT  system? 
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Facultv  Acceptance  and  Teacher  Bnlo 

°  ^CC^yp^gra^?8!  18        a"ltude  of  faculty  »embe"  toward  the 

°  progr^?"  aCtiVitie8!  18  the  role  of  an  instructor  in  the  TICCIT 

Implementation 

o  Project  administration:    What  mechanisms,  plans,  andVtivities  were 
involved  in  TICCIT  implementation?    How  did  courseware  production  take 

o  Site  management:    What  does  site  management  entail  as  exemplified  at  the 

toTSS^?  X*  demon8tra5i0n  8ites?  Mhat  chan*es  t00k  Place  i»  order 
to  facilitate  the  introduction  of  the  TICCIT  program  to  the  colleges? 

Instructional  Conditions 

c 

o  Context  for  the  demonstration:    What  is  TICCIT?    Wha^  constitutes  the 

Irrrj?*  8tandard  for  th*  comparison  of  outcomes  with  those  fltthe 
4  lxuuxi  program?^ 

As  should  be  obvious  from  these  questions,  there  is  substantial  overlap  in 
the  purposes  of  this  evaluation  and  the  courseware  design  goals  of  the^TICCIT 
program.    The  one  significant  point  of  departure  concerns  our  repeated  use  of  . 
information  from  regular  classes  as  a  relative  standard  for  comparisons,  espe- 
cially data  on  student  performance,  rather  than  rely  on  outcomes  from  TICCIT' 
classes  in  isolation.    Such  standards  were  indispensable  in  gauging  the  effects 
of  the  TICCIT  program:     the  results  from  regular  closes  became  our  benchmarks 
in  determining  whether  an  effect  was  positive  or  negative.    Indeed  the'evaluator 
seldom  accepts  expectations,  whether  those  of  the  developer  or  others,  as 
standards  for  a  project's  success  or  failure.    This  evaluation  points  to  relative 
strengths  and  weaknesses  of  the  TICCIT  program  in  contrast  with  the  educational 
outcomes  obtained  through  traditional  practices.  ,  ^| 
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The  chapters  in  this  report  correspond  to  the  questions  raised  in  thp  ■> 
evaluation.    Separate  chapters  deal  with  course  completion  rates,  achievements, 
ettitudes,  and  activities  as  components  of  student  performance.    The  analyses 
for  characteristics  that  might  distinguish  among  students  in  theip  performance 
appear  within  the  chapter  for  the  outcome  In  question.,  A  chapter  on  faculty  „ 

acceptance  and  teacher  role  comes  after  the  ^treatment  pf ^student  performance. 

'    '     "••    .  •  '  '•-  '  * .  •  •  - 

Since  TICCIT  represents' an  .innovative  curricular  prxigram,  An -early  chapter 

about  implementation  of. the  TIcfclT  program  attempts  to  d^cument^highlights  of 

its  development  and  introduction  into  schools  as  we^l  as  experiences  in,  its 

>  first- year*  of  'operation  at  community  colleges.,  -^description  of  theconditions 

across  ^aarison  groups  for  the  evaluation  is  g>ven  in  this  same^ chapter  in  / 

order  to<*set  the  contact  for'  the  Ticm^demonsttation.r  '  V,'  '  S* 


V    -  'Ihalt^ekonstration,  and"  tience  our  evaluation,  "began  after  the  T1CC1T  prb- 

4  '.*     .1.  w""  ^"^"  I  .  K  .... 


gram  W  into,  the  colleges''  sfrruc ture  and  when  the  developers  f kit 


confident; 


a  mere 
time  of 


^^n^h^o'en^orSe^th'^ICC^^rx)g^^8  a  pr^d^ct^ather  th>a 
,        prototype.  Over,  a  year  of  pil/>t  ^trlal^at  the  colleges  and  even  more  td 

-  •  system  work*  aJproducfrion  sltSs'.precede^ou*,  evaluation  of  the  TICfclT  n.rogr' 
A    'While  one  advantage  of  instructional  technology,  is  the  felative  ease  of 

adap'tationand  refinement  the  results  from*'  the  demonstration  period  shou" 


:  V    apply  to  similar,  uses^Me  TlG<3fl^#&  and  forms       instruction  close  to 
NriCCnkin  des'£|n  ?nd( 

....  .,  .  •«  ••-*AT 


rbf^app^catied.    fhe  nature)of  t\e  TICCIT  program 


and '  the  Vo'tls isteticy  of  ouAe indings  lead  us  to  believe  that  this  report 


presents  an  accurate  picture  of  the  T1CC1T  program's  effects  o\  instruction. 
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Matters  Qntaide  the  Scope  of  This  Evaluation 

While  this  report  touches  on  costs  and  system  performance  at  certain 
points,  the  evaluation  here  focuses  on  educational  issues.    There  is  no  com- 
prehensive treatment  of  matters  related  to  costs  of  the  T1CCIT  program  or  the 
technical  capabilities  of  the  TICCIT  computer  system.    Nor  does  this  report 
record  the  history  of  the  TICCIT  project,  either  as  charts  of  productivity 
or  schedules  of  progress  and  slippage.    Rather  this  evaluation  constitutes  a 
comparative  field  study  of  the  «ducational  ^pact  of  the  TICCIT  program. 

It  is  also  important  to  note  limitations  in  the  scope  of  this  study. 
There  is  no  definitive  contrast  provided  here  between  TICCIT  and  alternative 
methods  fo*  teaching  on  an  individual  basis.    Nor  did  the  evaluation  attempt 
to  separate  effects  due  to  hardware,  software,  and  courseware  since  the  TICCIT 
program  integrates  all  three  components  into  one  curriculum.    Finally,  cautions 
arise  due  to  the  context  of  the  demonstration,     it  took  place  in  courses  for 
precalculus  math  and  English  composition  at  community  colleges.  Inferences 
about  probable  outcomes  in  other  subject  areas  or  at  different  levels  of 
education  would  be  premature. 


Si 


-  Chapter  2 
Methods  and  Measurement 

An  evaluation  of  a  progfitfjls  ambitious  as  the  TICCIT  project  tends  also 
to  be  complex  in  its  range  of  activities.    Much  careful  planning  went  into  the 
design  and  conduct  of  the  evaluation  so  that  it  would  both  probe  critical  issues 
in  depth  and  cover  the  breadth  of  probable  effects.    Since  we  as  evaluators  were 
involved  in  the  early  phases  of  the  TICCIT  project,  we  had  a  unique  opportunity 
to  follow  the  program  from  its  inception  to  its  demonstration.    And  the  evaluation 
benefited  from  this  early  exposure,     indeed  the  developers  saw  the  evaluation  as 
an  integral  component  of  the  project  and , cooperated  fully  in  its  conduct.  Hence 
we  were  able  not  only  to  lay  careful  plans  but  also  to  follow  them. 

This  chapter  is  about  the  evaluation  itself.    The  first  section  deals  with 
the  basic  questions  posed  in  the  course  of  the  evaluation  as  well  as  how  we 
planned  to  address  them.    Subsections  of  the  first  section  correspond  to  chapters 
in  this  report.    The  next  section  is  the  instruments  constructed  for  the  evaluatio 
Its  organization  reflects  the  major  questions  which  the  evaluation  sought  to 
answer.    Given  the  issues  and  the  instruments  it  is  natural  to  move  next  to  the 
procedures  for  data  collection.    The  methods  for  data  analysis  appear  as  the 
fourth  and  final  section  of  this  chapter.    The  order  of  chapter  sections  simply 
follows  the  order  of  stages  in  evaluation  and  does  not  imply  relative  importance. 
For  we  view  each  of  these  stages  as  equally  critical  in  that  the  strength  of  the 
evaluation  depends  on  the  separate  strength  of  design,  instrumentation,  data 
c  (Section,  and  data  analysis. 


2. 1    Design  for  the  Evaluation 

The  nature  of  the  TICCIT  program  determined  much  of  the  design  for  the 
evaluation.    This  is  as  it  should  be  for  without  a  design  sensitive  to  the 


treatment  there  would  be  little  chance  of  capturing  its  impac  J^»t  it  was  also 

important  to  avoid  bias  toward  TICCIT  built  into  the  evaluation.    Often  different 

Utricular  programs  simply  lead  to  different  patterns  of  results  (see  Walker  & 

!  | 

Schaffara^ck,  1974).    Thus  an  evaluation  which  gave  undue  emphasis  to  one 
program's  objectives  might  be  expected  to  yield  a  significant  difference  in 
kvor  of  that  program.    Our  responsibility  was  to  design  an  independent  evaluation 
sensitive  to  TICCIT's  potential  while  fair  across  instructional  conditions  and 
responsive  to  information  needs. 

The  evaluation  rests  on  a  major  assumption:     that  TICCIT  represents  a 
curricular  program.    In  this  case  the  computer  is  not  merely  an  alternative 
delivery  mechanism  for  the  same  instruction.    Nor  is  the  use  of  CAI  a  supplement 
to  regular  classroom  work  and  therefore  an  addition  to  existing  courses. 
Instead  the  TICCIT  program  makes  full  college  courses  available  to  students  on 
an  individual  basis.    The  content  coverage  in  such  courses  has  to  satisfy  the 
guidelines  of  the  established  curriculum.    Otherwise  students  would  not  receive 
credit  for  their  work  on  .the  TICCIT  system  or  fulfill  the  prerequisites  for  more 
advanced  courses.    Classes  offered  as  part  of  the  TICCIT  demonstration  were 
curricular  alternatives  .to  conventionally  taught  sections  of  the  same  courses. 
This  evaluation  treats  sections  taught  by  an  instructor  in  a  classroom  format  and 
those  conducted  on  the  TICCIT  system  with  teacher  support  as  alternative  curricula. 

While  consistent  in  content  coverage,  TICCIT  obviously  differs  in  strategy 
and  delivery.    The  learner-control  courseware  and  the  computer  configuration 
with  television  receivers  represent  bold  innovations  in  education.    Yet  neither 
is  a  self-sufficient  curriculum:    each  is  part  of  the  TICCIT  program.  Therefore, 
the  evaluation  addresses  TICCIT  as  a  whole.    No  attempt  was  made  to  distinguish 
.  between  courseware  and  hardware  or  to  attribute  effects  to  one  component  of  the 
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TICCIT  syitn  or  another.  Such  distinction,  would  run  counter  to  the  integrated 
design  of  the  TICCIT  program  and  require  manipulation  of  school  practices  beyon! 
the  scope  and  intent  of  this  evaluation. 

It  should  be  clear  that  this  is  a  curriculum  evaluation.    The  alternative 
curricula  happen  to  differ  along  several  major  dimensions,  including  their 
strategy  for  teaching  and  mode  of  instruction.    But  the  student  learning  that 
takes  place  within  a  course  should  satisfy  the  same  minimal  criteria.  Compar- 
isons of  TICCIT  with  similar  delivery  mechanisms  or  contrasts  across  different 
instructional  strategies  for  courseware  would  most  certainly  be  helpful  in 
selecting  among  such  alternatives.     S^ill^a  consumer  might  view  sue*  studies 
as  limited  in  their  value  if  none  of  thTaT^rnatives  matched  or  exceeded  the 
results  obtained  through  traditional,  proven  practices.     Thus  we  concentrated  on 
documenting  TICCIT's  effects  relative  to  the  outcomes  of  usual  lecture-discussion 
classes.  * 

f 

2.1. a    Focus  of.  the  Evaluation 

It  is  common  to  judge  the  quality  of  an  instructional  program  by  its 
effects.    This  places  emphasis  on  a  demonstration  of  results  rather  than  just 
an.  indication  of  potential.    We  expect  such  results  will  be  evident  among 
students  as  the  group  most  affected  by  an  instructional  program.     But  teachers, 
too,  bear  the  impact  of  curricular  innovation  through  changes  in  their  duties 
and  responsibilities.     These  two  groups,  students  and  teachers,  were  the  source 
of  our  data  about  the  TICCIT  demonstration. 

Prom  students  we  sought  data  on  achievement,  attrition,  attitudes,  and 
activities.    Faculty  at  the  participating  colleges  provided  us  with  their 
reactions  to  and  judgments  of  the  TICCIT  program,  as  well  as  reports  on  their 
professional  role  in  TICCIT  classes.     These  data  reflect  the  basic  questions 
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about  student  i»rfor«.nc.  and  faculty  .cc.pt.iic.  •ddr^Md  in  the  .v.lu.tion. 

Wh.t  h.pp.«»  to  student,  and  faculty  *.n  th.  TICCIT  co«put.r-...ist.d  in.truc- 

tion.1  sysfm  becomes  an  oper.tiou.1  p.rt  of  community  college  instruction? 

Student  Achievement.    Perh.pe  the  single,  most  important  criterion  in 

judging  a  program's  success  is  its  effect  on  student  achievement.  Educators 

tend  to  look  at  achievement  results  before  other  outcomes,  and  most  new  curricula 

come  with  .  promise  to  improve  achievement  in  some  manner.    Certainly  there  was 

interest  in  whether  the  TICCIT  program  would  result  in  higher  test  performance 

than  that  associated  with  usual  classroom  practices.     Its  developers  hoped  that 

*  -  r 

TICCIT  would  lead  to  mastery  of  the  subject  matter  for  85X  of  the  students 

enrolled  in  mainline  courses.    Achievement,  then,  did  receive  the  most  attention 
in  the  evaluation. 

Since  the  TICCIT  program  supports  full  college  courses,  it  was  natural  to 
neasure  achievement  at  the  start  and  end  of  a  course.    The  pretests  at  the  start 
of  a  course  gave  us  a  measure  of  students'  entrance  ability.    The  posttests 
indicated  how  much  students  had  learned  in  their  courses,  and  often  instructors 
used  them  as  final  examinations.    Our  objective  was  to  compare  posttest  perfor- 
mance, adjusted  for  entrance  ability,  between  students  in  TICCIT  classes  and 
students  in  usual  sections  of  the  same  course.    If  there  was  a  significant 
difference  in  their  overall  learning  within  a  course,  and  if  we  succeeded  in- 
eliminating  or  adjusting  for  extraneous  factors,  then  that  difference  was  a 

result  of  the  TICCIT  program. 

The  instructional  design  of  the  TI$p. program  meant  that  it  might  affect 
aspects  of  student  achievement  other  than  those  reflected  on  a  posttest.  TICCIT 
courseware  has  a  highly  modular  structure.    When  a  student  completes  a  unit,  he 
has  mastered  that  material  and  need  no  longer  refer  to  it.    In  contrast,  an 
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ins true  tor  sight  Imvi  a  topic  and  us.  latar  elaaaaa  to  review  it  or  suggest 
ite  exteneion.    If  either  a  etudent  forgot  material  from  TICCIT  or  a  student 
picked  up  concepts  from  repetition  in  lectures,  then  posttest  results  tell  an 
incomplete  a to ry.    Measures  taken  at  the  time  a  student  completes  initial 
coverage  of  a  topic  might  reveal'  differences  which  do  not  show  on  final  examina- 
tiona.    Where  course  structure  permitted,  we  used  topical  tests  as  measures  of 
immediate  learning  to  supplement  the  information  available  from  posttests. 

There  were  other  features  of  TICCIT' s  design  that  had  the  potential  to 
affect  achievement  in  another  way.    A  student  on  the  TICCIT  system  could  proceed 
at  a  rate  suited  to  his  speed  in  learning.    Also  he  could  take  advantage  of  a 
machine's  patience  for  repetition  and  drill.    TICCrEgftft earner  control  further 
expands  the  range  of  choices  available  to  a  student^fth  such  an  opportunity 
a  student  might  master  and  overlearn  material.    Indeed,  a  study  of  programmed 
learning  suggests  that  learner  control  can  facilitate  retention  even  if  no 
effect  on  acquisition  is  apparent  (Newkirk,  1973).    We  made  special  efforts  to 
track  students  across  academic  terms  so  that  tests  administered  in  subsequent 
courses  might  serve  as  measures  of  retention. 

Thus,  the  evaluation  covers  three  critical  components  of  student  achievement 
(1)  overall  learning  as  measured  by  final  course  examinations;  (2)  immediate 
learning  as  assessed  in  topical  tests;  and  (3)  retention  as  reflected  by  tests 
given  in  later  courses.    Math  courses  lend  themselves  to  such  a  testing  scheme: 
each  course  includes  several  topics,  lik>e  units  in  TICCIT  courseware  or  chapes 
in  a  textbook,  and  often  two  or  more  courses  make  a  sequence  spanning  introduc- 
tory, algebra  to  calculus.    Courses  in  English  composition  do  include  posttests, 
but  they  lack  the  discrete  topics  common  across  classes  that  would  be  necessary 
for  topical  testing.    Further,  a  sequence  of  English  courses  follows  from  stude 
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iattratt  In  different  literary  forms,  such  as  tha  short  story  or  poem,  rathar 
than  fro*  logical  content  interreiationehipe.    So  the  •valuation  had  to  concen- 
trate on  eamplea  of  the  critarion  task  in  composition  courses.    We  took  writing 
•amp lea,  as  well  "^administered  multiple-choice  tests  of  writing  skill*,  at  tha 


start  and  and  of  English  courses. 

Course  Completion  Rates.   <We> usually  think  of  course  completion  rates 
in  a  negative  sense  aar  attrition:    what  proportion  of  the  students  dropped  out 
'or  otherwise  failed  to  complete  their  studies?    In  this  report  we  prefer  to 
use  attrition's  positive  complement,  simply  the  proportion  of  students  who  do 
receive  course  credit.    This  estimate  is  more  consistent  with  a  view  of  education 
as  a  productive  system.     Indeed,  Irf  we  consider  achievement  to  be  a  measure  of  a 
program' 8  quality,  completion  rates  indicate  a  program's  success  in  terms  of 
quantity.    Perhaps  a  math  department  would  sacrifice  a  few  joints  on  a  final 
examination  for  a  curricular  program  that  increased  its  course  completion 
rates.    Productivity  in  higher  education  is  really  a#ombination  of  student 
.achievement  and  course  completion  rates. 

Unlike  students  at  lover  levels  of  education,  college  students  can  give 
concrete  expression  to  their  dissatisfaction  with  instructional  conditions  or 
disillusionment  with  their  own  performance  by  formally  withdrawing  from  a  course 
or  simply  failing  to  appear  for  classes.    This  isv an  obvious  form  of  attrition. 
Completion  rates  are  subject  to.  subtler  influences,  especially  with  instructional 
programs  geared  for  the  individual  student.    A  student  may  continue  to  attend 
classes  regularly  but  still  not  complete  the  amount  of  material  required  for  m 
course  credit.     That  student  would  receive  a  $rade  of  incomplete  or  anoffier 
grade  which  indicated  that  he  had  yet  to  finish  the  course.    We  can  adjust  our 
estimate  of  completion  rate  so  that  it  is  sensitive. to  sufch  incidents. 
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This  report  prssents  three  different  estlmetee  of  the  collation- rata 'for 

TICCIT  and  lactura  classes.    Tha  thraa  corra.pond  to:     (1)  the  proportion  of 

students  enrolled  for  the  course  who  receive  course  credit^(2)  the, san^pro- 

portion  corrected  for  early  course  withdrawals;  and  (3)  the  proportion  of 

students  who  make  satisfactory  course  progress  to  the  total  course  enrollment. 

Each  of  these  figures  (examples  and  further  explanations  can  be  found  in 

Chapter  4:     Course  Completion  Rates)  reveals  something  different  about  the 

TICCIT  program.     We  begin  to  see  how  an  innovative  curriculum  project  affects 

v  t 
an  institution's  productivity.    Marked  contrasts  between  an  individualized 

program  and  a  group  teaching  method  also  emerge.    The  direction  of  these 
contrasts  depends  on  how  we  choose  to  define  completion. 

All  of  our  definitions  for  completion  rely  on  grades  given  at  the  end  of 
an  academic  term,     if  the  amount  <£  time  a  student  needs  to;,  master  an  entire 
course  does  depend  on  his  entrance  ability  (see  Bloom,  1974;  Carroll,  1963), 
then  an  academic  term  represents  an  arbitrary  unit  of  time.    More  able  students 
will  finish  sooner  while  students  less  proficient  in  terms  of  their  initial 
skills  may  take  longer.    So  it  was  important  that  the  evaluation  track  not  only 
course  completion  within  an  academic  term  but  also  enrollment  in  subsequent  ' 
courses  across  terms.     Enrollment  trends  across  terms  might  reveal  effects  hidden 
by  completion  rates  taken  within  an  academic  term.     From  enrollment  figures 
matched  with  the  instructional  condition  from  previous  courses  we  can  check  on 
whether  an  introductory  algebra  course  on  TICCIT  leads  students  to  continue  with 
intermediate  algebra.    Such  a  positive  effect  on  students^  progression  through 
a  course  sequence  should  take  place. . Otherwise  teachers  displaced  from  intro- 
ductory courses  would  find  no  teaching  demand  a*  a  counterbalance  (unless  the 
TICCIT  program  instead  attracted  additional  students  to  the  introductory  courses). 


Just  as  completion  and  enrollment  interact  with  college^pjroductivity  and 
teacher  role,  so  a  variety  of  factors  really  come  together  in  these  student 
counts.    Completion  rates  reflect  students'  academic  progress,  their  own 
satisfaction  with  that  progress,  extracurricular  commitments  like  family  or 
employment,  as  yell  as  possible  variations  due  to  subject  matter,  level  of 
course,  or  teacher.    Where  possible  we  collected  information  on  these  variables 
so  that  we  could  isolate  the  effects  of  the  TICCIT  program.    But  it  is  helpful 
to  think  of  completion  rates  and  enrollment  trends  as  measures  of  student 
performance  that  become  evident  at  the  level  of  the  institution.    While  college 
administrators  might  not  see  TICCIT  results  in  terms  of  posttest  scores  or 
attitudinal  responses,  they  will  receive  enrollment  summaries  and  grade  dis- 
tributions.   We  simply  made  that  level  of  TICCIT' s  impact  a  routine  part  of  the 
evaluation. 

At  the  time  of  our  initial  plans.  (Alderman  &  Mahler,  1973),  we  did  not 
know  the  importance  that  completion  rates  would  assume.    For  should  a  curricular 
program  lead  to  a  student  population  completing  th&Lr  studies  that^ln  some 
way  different,  from  the  reference  group  under  another  program,  all  metres  taken 
at.  the  end  of  a  course  incorporate  this  bias.    Statistical  analyses  can  compensate 
for  systematic  bias  but  only  in-1  a  partial  and  imperfect  sense.    Completion  rate  ^ 
would  then  "represent  the  only  tru^kndicator  of  program  result*. 

Student  Attitudes.    Over  the^ast  several  years  it  has  become  common  for 
students  to  rate  their  teachers  (e.g.,  Br/wn,  1976;  Costin,  Greenough,  &  Menges, 
1971).  -  This  practice  gives  impli'cit  recognition  to  the  role  of  students  as 
judges.    It  also  extends  the  criteria  for  teaching  effectiveness  beyond  achieve- 
ment  and  productivity  to  results  in  the  form  of  student  opinion.    Indeed,  studies 
of  computer-assisted  instruction  use  student  attitudes  to. explore  the  advantages 
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of  CAI  (e.g.,,  Mathis,  Smith,  &  Hansen,  1970;  Summerlin,  1971).    Despite  the 
acceptance  of  student  ratings  and  the  study  of  student  attitudes  toward  various 
teaching  methods,  curriculum  evaluations  seldom  compare  programs  along  this 
dimension.    Yet  comparisons  of  student  attitudes,  especially  with  dissimilar 
programs,  offer  us  insight  into  how  students  react  to  teaching  methods  and  what 
they  value  in  their  instruction. 

As  with  achievements  results,  an  evaluation  of  student  attitudes  requires 
a  relative  standard.    Where  there  were  other  sections  of  the  same  courses,  we 
had  access  to  such  a  standard  for  gauging  the  affective  outcomes  in  TICCIT 
classes.    Again,  then,  our  estimate  of  a  strength  or  weakness  in  the  TICCIT 
program  depended  on  what  results  already  occurred.    But  instruction  in  lecture- 
discussion  section!*  involve*  teachers  and  textbooks;  instruction  in  TICCIT 
classes  involves  television  receivers  and  keyboards.     Obviously  we  had  to  seek 
their  common  denominator  if  we  wanted  to  make  comparisons  across  conditions. 
Neither  rating  teachers  nor  assessing  student  acceptance  of  CAI  would  contribute 
to  our  evaluation  of  the  TICCIT  progral  as  a  curricular  alternative. 

Our  attitude  survey,  instead,  posed  questions  about  students'  satisfaction 
with||teir  class  and  about  the  usefulness  of  important  features  in  their  in- 
8      SBr "  the  m°8t  Part  the*e  Questions  dealt  with  aspects  of  ^l^^rse 

common  to  both  TICCIT  and  lecture  sections.     So  students  resiionded^ffl^dentical 
statements.    Where  *  specific  TICCIT  feature  had  a  strong' parallel,  ^regular 
classes,  we  drew  a^correspondence  between  them  and  considered  the  statements 
identical  for  purposes  of  analysis.     Two  items  from  the  survey  illustrate  this 
point:    all  students  indicated  their  extent  of  agreement  with  ^Instruction 
in  this  class  met  my  own  particular  needs";  students  in  TI^JfT  classes  also 
responded  to  ^JlCCIT's  comments' on  my  work  (ADVICE)  h^fiS|me  to  progress 
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^(through  this  course"  while  those  in  lecture  sections  reacted  to  "The  instructor's 
comments  on  my  work  helped  me  to  progress  through  this  course."    The  latter  pair 
of  ststements  reflects  the  stand-alone,  mainline  design  of  the  TICCIT  program; 
teachers  would  provide  help  with  planning  and  selecting  strategies  for  student 
learning,  hut  the  system  itself  would  be  the  primary  resource  for  advising 
students  on  their  course  progress  and  guiding  them  thtough  the  curriculum. 

Experience  in  pilot  trials  for  the  evaluation  had  taught  us  two  important 
points  about  student  attitudes.    The  first  was  that  students  viewed  the  TICCIT 
program  as  distinct  from  computer-assisted  instructional*,  was  much  easier  for 
them  to  respond  to  statements  about  TICCIT  in  particular  than  to  make  judgments 
about  CAI  as  a  generic  whole.    Thus  the  statements  in  |>ur  survey  refer  to 
TICCIT,  and  we  draw  no  broad  conclusions  about  studentx.attitudes  toward  ®AI  in 

this  report.    The  second^p^irit  was  closely  related:     students  had  difficulty 

J 

reacting  to  statements  about  CAI  when  they  had  no  prior)  exposure  to  computers. 
Confronted  with  a  questionnaire  about  their  predispositions  toward  CAI,  students 
tended  to  repeat  what  they  had  heard  in  their  orientation  to  the  TICCIT  system. 
There  was  no  apparent  mechanism  in  evaluating  attitudinal  impact  that^would  be 
comparable  to  an  achievement  pretest.    Our  approach  to  this  issue  relies  heavily 
on  straightforward  contrasts  between  responses  from  TICCIT  sections  and  those 
from  regular"  sections  of  the  same  course. 

Student  Activities.    Results  in  tfcrms  of  student  achievement,  course 
completion  rates,  and  student  attitudes  describe  what  happens  to  students.  But 
what  do  students  actually  do  on  the  TICCIT  system?    How  would  students  use  the 
learner  control  options?    Would  the  amount  of  time  required  to  attain  mastery  be 
a  function  of  students'  entrance  skills  in  the  subject  matter?    These  questions 
suggest  the  uncertainty  involved  with  the  first  field  trials  for  the  TICCIT 
program.    The  concepts  behind  the  program  were  innovative  and  experimental,  and 


-45- 


hence  how,  or  indeed  if,  students  would  exercise  learner  control  with  mainline 
instruction  was  an  open  <juestion.  ,  a 

■>k4£ 

Unlike  our  comparative  approach  to  evaluating  other  components  of  studSt 
performance,  our  emphasis  for  documenting  student  activities  fell  primarily 
within  the  TICCIT  condition  of/  a  course.     At  first,  field  representatives 
working  for  the  evaluator  made  direct  observations  of  the  interaction  between  a 
student  and  the  TICCIT  system.    These  laborious  observations  were  replaced  by 
automated  data  collection  on  the  TICCIT  system  when  the  necessary  programs 
became  available.     None  of  lectures,  class  discussions,  or  even  programmed 
textbooks  had  a  counterpart  to  the  individual  student  sessions  that  take  place 
at  a  TICCIT  terminal.     So  our  study  of  these  interactions  necessarily  focused  on 
issues  within  TICCIT  classes.    We  chose  to  address  (1)  students'  use  of  learner 
control,  (2)  sequences  and  dimensions  which  describe  that  usage,  and  (3)  the 
relationship  of  mastery  to  study  time  and  entrance  skills.     From  such  a  focus  we 
should  be  able  to  determine  whether  the  TICCIT  system  promotes  and  accommodates 
different  learning  strategies. 

Still,  there  were  questions  about  student  activities  that  require  a  relative 
standard.     If  the  TICCIT  program  had'  an  effect  on  the  frequency  of  students' 
interpersonal  contacts  or  the  efficiency  of  students'  course  work,  these  effects 
would  be  evident  only  in  contrast  to  the  contacts  and  study  requirements,. of 
other  course  sections.    We  relied  on  student  self-reports  to  give  us  estimates  of 
both  the  extent  of  personal  interaction  and  the^time  spent  on  a  course.  Thus, 
students  estimated  the  frequency  of  their  course-related  contacts  with  other 
students  as  well  as  with  the  instructor  for  their  class,  and  the  number  of  hodrs 
per  week  devoted  to  class  attendance  and"  homework  as  the  two  primary  time 
demands  of  a  course. 
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Admittedly,  a  self-report  is  not  the  most  accurate  way  to  track  student 
activities.    It  doej|^  however,  allow  an  evaluator  to  obtain  an  indication  of  a 
significant  difference.    For  should  there  be  a  substantial  change  in  his  routine 
stud*  habits,  a  student  should  notice  it.     If  the  difference  were  so  subtle  as 
to  Require  that  the  evaluator  monitor  students'  activities,  perhaps  then  its 
diction  consumes  more  in  evaluation  resources  than  it  justifies^    We  should 
still  remember  that  the  study  burden  imposed  by  an  instructional  method  is  a 
real  cost,  if  not  for  us  then  for  the  students  who  must  spend  the  time. 

Faculty  Acceptance  and  Teacher  Role.    There  was  as  much  uncertainty  about 
what  instructors  would  do  in  TICCIT  classes  as  about  student  activities.     If  the 
TICCIT  program  succeeded  in  teaching  students  and  enabled  them  to  adjust  their 
own  instruction  to  fit  individual  needs,  what  then  would  be  the  teacher's  role? 
The  developers  of  the  program  foresaw  a  displacement  of  teachers  from  intro- 
ductory  math  and  composition  courses.     Instructors  would  assume  new  roles  as 
master  teachers  in  advanced  courses,  as  subject  matter  experts  on  teams  engaged 
invcourseware  production,  and  as  researchers  jconcerned  with  concrete  instruc- 
tional problems  (see  Bunderson,  1975).    But  such  role  changes  were  beyond  the 
scope  of  the  TICCIT  project  and  demonstration. 

Within  the  period  „of  the  demonstration  the  developers  hoped  to  prepare 
faculty  for  a  shift  in  their  teaching  responsibilities  and  classroom  duties. 
The  TICCIT  system  was  to  fulfill  the  usual  teaching  role:     it  would  convey  and 
explain  material  as  well  as  facilitate  student  learning  through  its  advice.  The 
teacher  in  a  TICCIT  class  would  function  as  a  manager-adviser,  helping  students 
to  make  choices  and  plans  for  their  learning  on  the  computer  system.     So  the 
teacher  would  become  another  resource  available  to  students  rather  than  the 
central  figure  in  instruction. 
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^     •  S^ch  changes  in  ttfe  teacR«t'6  roi^plpt  appear  byarly  ambitious,  either 
from -the  pefrspective/of  a  doubtful  faculty  member  or  frto  the  viewpoint  of  * 
of  t 


someoac^oavinced  of  the  conservative  nature  of.  education.    As  a  program  goal, 
however",  it^indicated  a  need  for  attention  to  t^afchers'  activities  in  the 
„  evaluation.     The  simplest  method  for  gathering  data  on  teacher  duties  was  to 
distribute  a  questionnaire  to  all  instructors  responsible  for  sections  of  target 
coutses.     That  questionnaire  required  teachers  to  estimate  the  class  time  spent 
in  different  teaching  modes  (e.g.,  lecture^discussion,  programmed  textbook, 
TICCIT  system,  tests  and  quizzes)  each  week,  and  their  own  alldfcfction  of  time  as 
a  percentage  distribution  among  various  teaching  duties  (e.g.,  planning  for 
class,  preparing  lectures,  conducting  class).     Each  Section  assignment  called 
for  separate  estimates,  with  general  faculty  duties  unrelated  to  specific 
sections  covered  on  another  part  of  the  questionnaire.  \ 

Two  other  sources  lent  descriptive  support  to  the/teachers'  activity 
reports.     Records  of  students'  requests  for  assistance,  kept  as  part  of  our 
observations  of  student  interactions  with  the  TICCIT  system,  and  class  obser- 
vations undertaken  for  both  lecture  and  TICCIT  sections,  helped  us  to  appreciate 
what  instructors'  responses  meant.  \ 

Proponents  of  CAI  cite  school  reactance  as  one  reason  for  the  slow  spread 
of  instructional  technology.     This  must  be  at  least  partially  due  to  faculty 
acceptance.     To  assess  faculty  positions  on  the  assumptions  behind  TICCIT  and  to 
track  their  reactions  to  the  program,  the  evaluation  included  a  series  of  three, 
annual  attitude  surveys.     These  faculty  surveys  also  gave  us  access  to  expert 
judgments  in  the  form  of  educators'  opinions,  the  opinions  of  those  teachers 
closest  to  the  project. 
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2.1.b    Comparison  Groups 

It  should  be  obvious, , after  so  much  repetition  of  the  point  throughout 
the  discussion  of  student  performance,  that  the  evaluation  of  the  TICCIT  program 
depended  on  comparison  groups.    For  each  target  course  in  the  demonstration  there 
had  to  be  both  classes  offered  on  the  TICCIT  system  and  sectioirf  taught  Uk 
traditional  classrooms.     In  one  case  there  were  also  sections  that  used  programmed 
instruction.     Such  alternative  programs  of  instruction  were  essential  if  we  were 
to  evaluate  th6  outcomes  of  the  TICCIT  program  in  relation  to  already  attainable 
results.    Without  a  relative  standard  there  would  be  no  way  to  know  the  difference 
between  a  strength  and  a  weakness  in  the  program,  and  thus  no  basis  for  future 
decisions  about  TICCIT. 

Actually,  it  is  not  quite  accurate  to  describe  the  TICCIT  program  as  an 
alternative  to  existing  courses.     This  implies  that  it  leads  to  similar,  if  not 
in  soriie  way  preferable,  results.     As  evaluators  we  believe  such  a  description 
premature  in  the  absence  of  ^iata.    What  the  TICCIT  program  really  represented  at 
the  start  of  the  demonstration  was  a  competing  curriculum.     Especially  with  an 
independent  evaluation  contracted  for  the  demonstration,  TICCIT  had  to  compete 
with  whatever  practices  were  already  in  place  at  the  colleges.     And  it  had  to 
produce  at  least  equivalent  results,  or  else  an  acceptable  balance  of  results, 
should  comparisons  across  outcomes  prove  uneven.     Otherwise  it  would  be  difficult 

r-, 

to  make  a  case  for  school  adoption  of  a  TICCIT  system. 

Our  treatment  of  completion  rates,  student  achievement,  and  student  attitudes 
rests  on  comparisons  made  across  teaching  methods.    ^4he  importance  of  these 
comparisons,  both  to  the  credibility  and  generalizability  of  the  evaluation  and 
to  the  participants  in  the  project,  justified  careful  attention.    We  did  exercise 
care  in  selecting  the  criteria  for  comparisons,  checking  the  equivalence  of 
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student  groups  at  the  start  of  an  academic  term,  and  looking  at  the  actual 
instructional  conditions. 

Criteria  for  Comparisons.    Wherever  possible  the  criteria  for  comparisons 
were  identical  across  course  sections.     For  completion  rates  this  meant  that  we 
considered  a  letter  grade  in  one  section  to  mean  the  same  thing  in  another 
section.    True,  one  teacher  might  mark  on  a  different  scale  or  distribution  than 
his  colleagues.    But  our  primary  interest  was  whether  a  student  earned  a  grade 
sufficient  to  receive  course  cr/adit,  not  how  high  a  grade  a  student  received. 
Th^re  were  departmental  policies  that  affected  grades  like  incompletes  and 
withdrawals,  so  the  danger  of  bias  due  to  individual  teachers  was  minimal. 
Stillt  a  collective  faculty  prejudice  might  cause  completion  rates  to  differ 
across  conditions.    As  evaluators  familiar  with  the  colleges  we  did  not  believe 
this  would  happen.    We  also  knew  that  such  prejudice  in  assigning  course  grades, 
if  it  existed,  would  show  in  our  analysis  of  achievement  results  as  an  effect 
in  the  opposite  directions 

The  need  for  a  common,  shared  criterion  is  perhaps  most  pronounced  when 
evaluating  results  in  terms  of>student  achievement.     Should  objectives  or 
content  coverage  differ  among  competing  programs,  this  discrepancy  rather  than  a 
real  difference  in  effectiveness  might  lie  behind  the  significance  of  *  statistical 
tests.     It  is  therefore  useful  to  restrict* the  basis  for  comparisons  to  or^y 
those  objectives  held  in  common  across  programs  (e.g.,  Popham,  1969).  Shoemaker 
(1972)  points  out  that  a^  the  number  of  programs  increases  and  as  it  becomes 
necessary  to  infer  objectives,  the  size  of  the  common  basis  must  shrink. 

With  just  two  or  three  alternative  methods  for  instruction  and  a  well- 
defined  content  domain,  there  was  substantial  overlap  in  objectives  for  math  - 
courses.     Indeed  all  sections,  regardless  of  method,  had  to  meet  the  same 
^pirriculum  standards.     So,,  with  the  cooperation  of  both  the  math  faculty  at  the 
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colleges  and  the  developers  of  the  TICCIT  math  courseware,  we  classified  each 
posttest  item  as  one  that  reflected  either  a  common  objective,  an  objective 
unique  to  one  program,  or  an  objective  beyond  the  course's  scope.    Our  comparisons 
of  achievement  results  in  math  courses  use  the  posttest  score  for  just  those 
items/  reflecting  common  objectives. 

/ TJie  statement  of  objectives  for  TICCIT  courseware  was  as  explicit  in 
English  composition  as  in  pre-calculus  mathematics.     Bi^t  there  was  a  variety  of 
approaches  among  instructors  on  how  writing  is  best  taught  to  students.  Thus 
there  was  disagreement  on,  for  example,  whether  a  student  had  to  know  about 
proper  use  of  the  semicolon,  in  order  to  write  w£ll.     This  lack  of  a  consensus 
within  the  English  departments  also  meant  that  it  would  be  quite  difficult  to 
specify  a  set  of  objectives  common  to  TICCIT  classes  and  lecture-discussion 


sections.     Still,  despite  differences  on  how  composition  should  be  taught,  ^he 

\ 

importance  attached  to  a  student's  actual  writing  was  not  subject  to  dispute. 
Instructors,  among  t^e^selves  or  with  the  developers  of  the  TICCIT  courseware, 
might  disagree  ^yjB^BRp^*^^  as  appropriate  course  objectives.  But 

the  overriding  objective Vn  the  composition  courses  was  clear  and  effective 
written  communication.    We  took  waiting , samples  from  students  as  the  best 
indicator  of  course-related  a^plevement . 

Unlike  the  explicit  statements  of  objectives  (or  implicit  in  content 
coverage)  associated  with  student  achievement,  curriculum  impact  on  student 
attitudes  was  left  unspecified.     Of  course  both  the  TICCIT  developers  and 
college  faculty  hoped  that  students  would  react  favorably  to r<heir  courses.  In 

»  « 

addition,  there  were  a  number  of  general  advantages  claimed  for  tte^ins true t ion 
available  with  the  TICCIT  program.     For  example-*- mainline  instruction  combined 
with  learner  control  was  to  make  students  more  responsible  and  more  actively 
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involved  in  their  own  learning.     It  would  also  permit  a  student  to  set  a  pace 
right  for  his  own  particular  abilities  and  to  receive  indi^dual  attention  from 
an  instructor.    -But  would  the  TICCIT  program  really  improve  the  extent  of  a 
student's  active  participation  in  his  learning,  and  to  what  extent  did  lecture- 
discussion  classes  already  meet  the  advantages  traditional ly  advocated  for  CAI?  * 
It  was  appropriate  for  the  evaluator  to  apply  the  same  criteria  across  sections 
and  instructional  conditions. 

i 

Whether  in  TICCIT  classes,  lecture  sections,  or  programmed  instruction  classes 
students  responded  to  essentially  the  same  attitude  survey.     The  survey  had  state- 
ments drawn  from  popular  claims  about  CAI  as  well^s  specific  features  of  the 
TICCIT  program.     It  dealt  generally  with  students'  satisfaction  with  their  course. 
For  each  statement  a  student  responded  on  a  scale  from  strongly  agree  (coded  5)  to 
strongly  disagree  (coded  1).    We  compared  results  from  TICCIT  and  lecture  classes 
in. terms  of  the  student's  attitude  score  for  about  20  items.     These  were  the  state- 
ments which  were  identical  or  closely  parallel  across  instructional  conditions. 

Formation  of  Groups.     Assessing  outcomes  in  terms  of  the  same  criteria  is 
one  step  in  program  evaluation:     assuring  the  equivalence  of  groups  exposed  to 
different  programs  is  another.     The  preferred  method  to  assure  equivalence  is  to 
assign  students  to  groups  on  a  random  basis.     Then  there  would  be  no  grounds  for 
belief  that  preexisting  differences,  like  entrance  ability  in  the  subjeq£  matter 
or  demographic  variables  such  as  sex  or  age,  affected  outcomesfin  one  group  in  a 
somehow  different  manner  than  outcomes  in  another  group.     As  an  extreme  case, 
consider  what  might  happen  if  all  students  motivated  to  complete  their  studies 
enrolled  in  familiar  lecture  se  (ions:     we  might  then  expect  results  for  Jlurse 
depletion  rates  to  favor  lecture  instruction  although  another  variable  was 
pc>tually  responsible  for  the  difference  between  comparison  groups. 
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Of  course  completely  random  assignment  was  inappropriate  for  the  TICCIT 
demonstration.     Such  a  procedure  would  in  effect  cancel  the  student's  prerogative 
to  choose  courses.    At. best  those  students  seeking  to  ^jiroll  in  a  target  course 
could  be  randomly  assigned  to  a  condition.    This  was  the  procedure  followed  at 
Phoenix  College,  with  the  consent  and  cooperation  of  math  faculty  members,  for 
the  fall, semester  of  1975-76.     But  more  often  the  constraints  imposed  by  regis- 
tration, class  time  schedules,  space  availability ,  and  faculty  or  staff  hours, 
make  random  assignment  unfeasible.     The  evaluator  cannot  enforce  experimental 
control  which  disrupts  college  operations  as  well  as  services  to  students.  Even 
when  we  succeeded  in  arranging  for  a  truly  random  procedure,  students  with 
strong  objections  were  allowed  to  change  their  assignment  to  an  instructional  • 
condition. 

An  evaluator  responsible  for  a  comparative  study  of  curricula  usually 
has  to  seek  an  alternative  to  random  assignment.     And  we  did  implement  an 
approximation  to  random  assignment  at  the  Alexandria  Campus  of ^ Northern  Virginia 
Community  College.     The  fall  and  winter  (1975-76)  schedules  for  classes  at 
Alexaridria  did  not  identify  which  sections  of  a  course  were  TICCIT  classes. 
Instead  the  schedule  listed  just  the  time,  room,  and  instructor,  along  with  the 
course  title  and  number.    Where  programmed  instruction  was  already  available  for 
a  course,  the  schedules  maintained  the  precedent  of  earlier  years  and  identified 
sections  as  "lectyre"  or  "programed. "    In  this  case,  the  TICCIT  classes  were 
actually  a  subset  of  those  "programmed"  sections.     It  should  perhaps  be  noted 
that  such  a  procedure  is  an  attempt  to  guard  against  the  formation  of  nonequiv- 
alent  groups  rather  than  an  explicit  method  to  creatli  equivalent  groups. 

We  had  taken  care  to  arrange  for  random  or  quasi-random  enrollment  in 
comparison  groups  so  as  to  eliminate  sources  that  might  contaminate  evaluative 
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results.    Campbell  and  Stanley  (1963)  note  the  strength  of  the  pretest-posttest 
control  group  design,  as  ours  would  be  for  achievement,  and  the  posttest-only 
control  group  design,  as  ours  would  be  for  completion  rate  and  utudent  attitudes. 
•Such  experimental  designs  presuppose  random  assignment  without  alterations  to 
groups  during*  the. period  of  the  study.     But  students  would  withdraw  from  courses 
or  otherwise  fail  to  complete  a  course.     Then  not  all  students  would  take  the 
achievement  posttests  or  attitude  surveys,  and  the  evaluation  design  for  these 
outcomes  would  degenerate  into  what  Campbell  and  Stanley  (1963)  call  anon- 
equivalent  control  group  design. 

Indeed,  we  found  pronounced  differences  between  the  number  of  students  who 
started  a  course  and  the  number  who  finished  it  With  credit.     This  fact,  together 

!,  -    •  /  ) 

with  preregistration  by  mail  for  the  second  term  and  student  wishes  to  reenroll 
,   in  a  math  course  under  the  same  condition  as  in  the  fall,  convinced  us  that  we 

•V, 

had  to  abandon  random  assignmenj^or  the  spring  semester  at  Phoenix.    While  the 
English  composition  course  at  Alexandria  continued  to  mask  the  identity  of 
classes  which  included  TICCIT  use,  math  courses  there  had  sections  listed  with  ' 
the  appropriate  instructional  condition  by  the  spring  quarter.     Again  the  change 
from  quasi-random  procedures  to  selection  at  the  student's  discretion. 

It  might  be  argued  that  such  student  selection  conforms  to  what  other 


colleges  do  and  hence  is  more  representative  than  a  forced  and  artificial  random 
procedure.     Or  that  college  registration  itself fis  a  quasi-random  procedure. 
However,  we  knew  a  nonequivalent  control  design  would  not  be  as  strong  as  a  true 
experimental  design.     It  does  not  permit  the  same  inferences  to  be  made  unless  - 

r 

we  eliminate  competing  explanations  for  group  differences.     These  sources  of 
invalidity  in  our  analyses  for  achievement  and  attitudes  should  show  first  under 
completion  rates. 
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If  there  were  a  differential  effect  on  course  completion,  involving  an 
interaction  between  class  condition  (i.e»,  TICCIT,  programmed,  lecture)  and  some 
student  characteristic,  then  the  students  who  had  t ate/ the  po'sttest  and  attitude 
survey  at  the  end  of  \he  course  already  constituted^^  dimension  of  educational 
vf^pact.    One  method  of  teaching  had  eliminated  certain  students  from  the  population 
taking  posttests  and  surveys  in  a  manner  dissimilar  from  that  of  another  teaching 
method.    Of  course  a  methtfd  might  also  permit  certain  students  to  complete  a 
course  while  another  would  not. '  The  point  is  the  same:    the  differential  effect 
would  be  built  into  the  analyses  for  achievement  and  attitude  results.  And 
further  interpretations  of  achievement  or  attitude  results  would  have  to  be 
cautious  and  conservative.     In  the  absence  of  differential  (not  just  different) 
effects  on  completion,  we  could  continue  our  analyseYof  TICCIT's  impact  on 
student  performance. 

With  nonequivalent  comparison,  groups  it  became  important  to  document  what 
the  groups  were  like  and  to  gather  additional  data  from  student*.  Information 
about  students'  reasons  for  selecting  a  particular  section  along  with  a  rough 
stu^t  profile  (e.g.,  age,  sex,  grade  point  average)  would  permit  closer  study 
of  the  comparison  groups  than  pretest  score  alone.     We  collected  this  background 
information  as  soon  after  registration  as  possible,  usually  on  the  first  class 
meeting  or  within  the  first  week  of  classes.     It  gave  us  data  on  why  student^ 
chose  particular  sections  of  a  course  so  tha^we  could  understand  the  process 
behind  group  formation.     It  also  provided  data  through  which  we  could  check  for 
differential  effects  both  as  a  threat  to  inferences  from  achievement  and  attitude 
results,  when  such  effects  arose  in  completion  rates,  and  as  evidence  that  one 
method  of  instruction  might  be  better  for  a  specific  subgroup  of  students,  while 
another  method  had  greater  success  vith  the  rest  of  the  student  population. 
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Context  for  Comparison  Groups.    An  assumption  perhaps  buried  in  the 
complexity  of  an  evaluation  is  that  the  alternative  curricula  represent  well 
defined  conditions.    We  too  made  this  assumption  in  developing  our  strategy  for 
data  analysis.    At  first  this  may  seem  an  unwarranted  leap  considering  the 
flexibility  of  the  computer  and  the  variety  of  teachers'  styles.    But  recall  that 
our  criteria  .for  comparisons  depend  on  objectives  common  across  groups,  especially 
wf|h  regard  to  student  learning.     So  there  were  specific  objectives  that  defined 
what  was  to  be  taught  in  a  class  and  thus  learned  by  students.    The  college 
curriculum  standards  further  specified  content  coverage  within  a  course,  and  our 
comparison  groups  furmed  within  each  course. 

Still  there  might  be  substantial  variation,  if  not  in  what  taught,  then 

in  how  it  was  taught.    Yet  there  was  an  obvious  difference  between  comparison 
groups  in  this  respect.     Those  classes  considered  part  of  the  T1CC1T  condition 
received  instruction  through  the  T1CC1T  System.   'Sections  included  under  the 
lecture  condition  were  taught  by  an  instructor  and  assigned  a  textbook.     The  one 
course  that  also  had  a  programmed  condition  actually  was  offered  with  programmed 
texts.     At  least  in  terms  of  the  primary  resources  for  teaching,  conditions  were 
well  defined.  . 

As  we  might  expect  from  the  dissimiliarities  in  the  subject  matters,  there 
.were  differences  between  the  math  and  English  courses  in  the  approach  taken  to 
teaching.     Math  courses  tended  to  be  highly  structured  and  sequential.  English' 
courses,  on  the  other  hand,  were  much  more  dependent  on  the  instructor's  profes- 
sional judgment  of  the  manner  best  suited"  to  teaching  composition  skills.  This 
is  not  meant  to  imply  a  stereotype  method  for  teaching  pre-calculus  mathematics, 
for  there  were  differences  in  strategy  if  not   in  overall  content  structure  and 
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sequence,    Subject  matter .dissii^larities  would  naturally  preclude  comparisons 
across  courses  but  should  not  interfere  with  comparisons  across  conditions  for  . 

r  /  .  .  .  ■#*• 

the  same  course-  -*  <r 

These  general  statements  about  comparison  groups  appear  n6w, perhaps  at  top 
early,  a  stage  in  bur  discussion,  so  that  we  can -begin  to  define  the  cont«*xt  of 
'     tW  TICCIT  ^^nstration  and  the  limits,  of  our  attention  to  the  instructional 
ptoci^  withiti  "feaclK  class  section.    We  sought  only  to,  cottfJ.rm  basics  exptct*tione 
/faout  what  happens  in  lecture  and  programmed  sections. .«  Through  occasional  . 
observations  and  contacts  with  instructors,  we,  came  to  know  that  a  class  did  ^ 
-indeed  belong  under  the  lecture  or  programmed  condition.    This  minimal  invest-, 
merit  of  resources  helped  isolate  exceptional  cases  that  might  later  surface  in 
analyses,  such  as  the  lecture  section  found  in  pilot  trials  in  which  the  instructor 
permitted  each  student  to  set.  his  own" pace  and  complete  the  course  prior  to  the 
end  of  the^  academic  term,  or  the  section  scheduled  to  lose.  TICCIT  in  the  demon- ,  ^ 
stration  period  which  did  not.    To  find  these  exceptions  and  get  a  better  descr^p- 
,  tion  of  cond^ions  were  out  modes t\purposes  in  classroom  observation.    We  did  not 
attempt  to  titTparticular  teaching  practices  with  'section  by^ection  results. 

There 'was  not  sufficient  precedent  for  us  to  .formSisfinite  expectations 
of  what  the  '  TICCIT program .would  be  like.     Here  we  did  want  to  document  what  * 
bapfened  in  a  class  and  again  describe  the  instructional  condition  as  a  whole. 
*But  with  the  use  of  the  TICCIT  system  often  at  the  discretion  of  the  instructor, 
particularly  where  the  TICCIT  program  sesved  as  the  laboratory  component  of  a 
'    composition  course,  it  was  necessary  to  supplement?  our  direct  observations. 
Rec9rds  of  students'  progress- in  the  TICjCIT  program  wye  available  on  the 
computer  system.    These  records  indicated  what  aaWs^ai  . a  student  had'  studied 
apd  mastered,  as  well  as  how  auchjtime  the  sturfent  had \ spent  on  the  system. 
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Online  data-  summaries  for  TICCIT  classes  ^MArvatipne  conducted  on  a  regular 
"*       b^f^^tdfour  primary  sources  for  deecript!^r±n£orniat.lon  on  the  nature  of  the 
.Tl$^pr<**am  a^^^opariBon  group.  *>  : 

c|^^Mj  observations  played  a  limited  role  in  our  evaluation. 
'-^»ii^j:i«^ner''c^Kf«ncet' keeping-,  detailed  section  histories,  and  drawing 
.  ' '■•  '^^^f  *bo*t  .-"the  instruction  within  a  single  class  all  go  beyond  the  scope 
.0-CT?^^*;tio?'    °Ur  observations  were  made  simply  "to  confirm  what  happened^ 
y  •  J^^l^^^'*^.^:-*^  to  describe  tfte  context  for  our  comparison  groups  in 
;f ■•."^Wi SSlPC?    "l^ college  standards  met  by  the  TICCIT  program  in  order  to  be 
t   *  ^f^f  ^*  Pfct  of  the  regular  curriculum  and  our"  emphasis  on  common  criteria 
?^VH,fcCOme  measures  already  defined  courses,  regardless  of  condition,  to  a 

extent.    And  the  TICCIT  program  was  itself  well  defined  from  the  standpoint 
^l^£culum  obJectives,  content  coverage,  and  teaching  strategy. 


9 
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2.?    Instrument  Preparation  and  Properties 

•'"   r  :    

Ask  an  evaluator  a  question  and  an  instrument  for  data  collection  is  apt  'to 
precede  Ms^answer.    At  least  thrae  major  options  were'  open  to  us  in  selecting 
instruments.    We  might  choose  to  measure  TICCIT' s  impact  through  standardized 
tests  pd  surveys.    Due  to  their  demonstrated  characteristics  and,  widespread 
use;  standardized  instruments  "offer  k  convenient  solution  for  obtaining' measures 
of  results.    But  such  a, test  is  of Cen  insensitive  to  the  particular  program 
under  study  in  that  its  match  with  the  curricular  coverage,  omits  much  detail 

1  .  '  ■ 

(Shoemaker,  1972)  and  its  aim  is  to  assess  a  student's  knowledge  rather  than  a 
program's  results.     Similarly,  surveys  of  attitudes  and "activities  broad  enouih 
to  be  available  on  a  commercial  basis  often  omit  the  very  question^  most  germane 
to  the"  particular  program.    We  also,  found  a  dearth  of  standardized  instruments 
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specifically  developed  for  community  colleges,  an  unusual  fact  given  their 

growth  rate  over  the  last  two  decade 9.  >  . 

■  -  *M 

Criterion-referenced  instrumenfti  offer  another  means  to  assess  a  program's 


impact.    Indeed  such  tests  would  be  consistent  with  several  of  TICCIT' s  goals,         ,  ># 
most  no  tab  ljr  that  85  percent  of  students  attain  mastery.    And  criterion- 
referenced  tests  correspond  to  the  curriculum  undjfcistudy .    However,  this  mode 
of  testing  Implies  a  firm  grasp  of  what  constitutes  mastery,  a  concept  that  still 
Is  subject  to  numerous  interpretations  ^ke,/1970),  and  bypafcses  measuring  the 
extent  to  which  material  is  learned  (Ebel,  1973).    Goals  o'f  the  TICCIT  program 
besides  student  achievement,  lite  those  for  improved  completion  rates  and  better 

kd  -  ■  '-L'"*9  - 

1  student  ^etif&des  toward  learning,  required  a  relative  standard  rathecr  t|ftp  an 
absolute  criterion.  '  -.  ^ 


A  third  tool  with  which  we  can  capture  the  impact  on  studenjf  performance 

and' faculty  a^fetance  is  special  teste  developed  for  the  specific  program's 

evaluation. flKs  is  the  option  weJ  pursued.     It  enabled  us  to  focus  our  achieve-  ^ 

ment  test  on  the  material  taught  in  target  courses  and  to  develop  questionnaires 

and  surveys  that  met  the  specific  needs  of  the  evaluation.    Special  tests  also 

permitted  us  to  engage  in  constructive  exchange*  with  the  developers  of  the  ■ 

•  TICCIT  courseware  and  faculty  members  at  participating  colleges,  and  to  make  >i*rg 

revisions  to  our  instruments  as  warranted  by  faculty  comment  or  data  from  pilot 

trials.    However,  there  is  a  burden  associated  with  these  benefits.  Besides 

-  devoting  resources  to  instrument  preparation,  an  e^aluator  must  demonstrate  the 
-# 

proper ties  of  specially  constructed  measures. 

2. 2. a  ,  Student  Performance  jgty 

The  processQf  preparing  instruments  particularly  suited  for  the  TICCIT 
demonstration  was  especially  critical  to  our  assessment  of  student  performance. 

•:    -  ■  '  •',  / 
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Not  only  did/we  need  fair.measures  of  achievement,  at'titudea/and  activities,  but 
we  also  had  to  develop  items,  conduct  pilot-  trials  and  revise  the  instrument. 


times  it  seemed  ^he 


ptior  to  use  during  the  demonstration  period, 

process  might  never  come  to  a  close.    There  were  changes  ln  the  colleges'  plans 
for  integrating  the  TICCIT  program  into  their  own  curriculum  structure  end- 
uncertainties  about  the  courseware  due  to  delays  in  its  production.    On  the 
other  hand  these  departures  from  the  project's  schedule  gave  utr  t 
field  trials  for  the  instruments  and.  refine  them.    When  the"TlSa^|rafio 
the  achievement  tests,  student  attitude  surveys,  and  activit^es^ia^ 


were  ready. 


What  follows  is  an  historical  afc'c 


icount  of  how  we  developej  £hes«y £ns|rume^ 
and  a  presentation  of  data  pertinent  to  their  reliability  a^^lld^tyi  Tn^ 
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section  does  not  convey  any  results  of  ^demonstration,     itVeparjs  only  tha 
data  necessary  to  demonstrate  instrume/t  properties.    We  might  be^pted  to,^ 
skip  much  of  the  detail  here  but  for  alekef  Chat  the  accuracj^o%  Ir  res»lS \* 
rests,  in  part,  with  the  instruments.  T.-  i"  ' 

1  .  5  "  # 

There  is  an  obvious  omission -from  this  account;",  complet 
college  grade  sheets  contained  the  data  about  course  completion.  j|A*t 
college  we  contacted  instructors  to  verify  what  different  grades,  li^; those 
for  withdrawals  and  incompletes,  meant.    There  were  also  different^lig* 
policies  from  term  to  term  and  different  standard^  "for  grading  f rdm  ilstract  ^ 
,-to  instructor.    But  we  generally  accepted  whatever,  grades  appeared  ^  tbe^^ 
rosters  as  the  most  meaningful  definition  o^oourse  completi^  These  grades > 
after  all,  determined  whether  students  received  academic  credits. 
•      Vr    Achievement  Tests,    Math.     Even  before  colleges  agreed  to  participate  ijf 
.the  demonstration  of  the  TICCIT  program,  the  process  of  test .construction  had 


> 


b«t#    SpeciaiUt^^veloping  math  teste  tfibt  revl«»^^e  of 
content  coverage  and Statement  of  objectives  in  1972.'  It  X^lear  then  that    :  > 
the  TICCIT  program  would  approximate  #pre-calculus  course  and  a  beginning  ' 
algebra  course.    The  pre-calculus  course  followed  the  recommendations  of  the  • 
Committee  on  the  Undergraduate  Program  in  Mathematics  (1969)  for,  a  Math  0  course 
that  met  the  requirements  for  transfer  to  |our-year  colleges.    The  beginning 
algebra  course  was  very  much  like  those  already  of  fered  ^community,  colleges  to 
7       meet  the- deficiencies  in  student  preparation  so  common ^under  open  enrollment. 
It  was  a  sound,  traditional  curriculum  plan.  . 
1  There  were  two  prominent  reasons  behind  our  e*rly  decision  to  prepare 

special  tests  \ or  the  math  component  of  the  TICCIT  program.    Obviously  a  . 
revolutionary  outline  of  contef?1  coverage  was  not  one^them.    Instead  it  was 
/  .       the  scarcity  of  commercial  tests  appropriate  for  commjjg  colleges, and  the  ^ 
f   :V*  insensitlvity  of  standardized  tests  to  specific  courses.    With  a  commltineni^de,    /  . 
1^.5^  engage  in  test  construction,  it  was  possible  to  anticipate  the  demands  of 
t*£ -ac\ivity\WJ>terms  of  the  evaluation's  schedule  and  resources. 


S^ne^alke  schedule  of  the  evaluation  had  to  follow  that  of  the  TICCIT 
'  P^^^^t  ^e  fir8t  ful1  ^ademic  yetr  of  the  evaluation,  1972-73,  passed 
^thS^t^ollection  but  with  much  progress  toward  satisfactory  achievement 
te8t,  inCmJth.    ^miners  (aS  we  prefer  .to  call  test  specialists)  reviewed  the 
..specifications  for  the  math  courseware,  "  When  two  colleges  were  selected  to 
serve  as  demonstr^rlon  sites,  the  same 'examiners  visited  the  math  departments 
and  <4,tai^*vTTO  syllabi  and  textbooks  used  for  algebra  and  pre-calculus  courses. 
This  infoLmation  was  sufficient  to  develop  a  pool  of  test  items.    WhfcLe  the   ,  . 

rV  cgfftinued  their  work  on  courseware  throughout  1973-74,  our- examiners 
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went  Vbve^  the  item  pool  with  both  the  math  faculty  from  the  participating 


colleges  anl  the  math  authors  from  the  courseware  production  team.  The  resulting 
posttests  were  given  field  trials  hafore  the  close  of  the  1973-74.  academic  year. 

&  Th«»  the  posttests  "for  algebra  courses  were  available  during  the  implementation  ' 
period  for  toe  TICCIT  project.    For  it  was  not  until  1974-75  that  T1CCIT  systems 
were  installed  at  the  college^  and  pilot  versions  of  the  algebra  courseware 
.tried  with  community  college  students. 

The.  pre-calculus  courseware  was  about  a  year  behind  the  production  schedule 
for  the  algebra  curricula.* Uncertainty  about  the  availability  of  TICCIT  pre- 
calculus  infts  and  an  abbreviated  trial  period  for  this  courseware  (one  academic 
term)  prevented  us  from  following  as  thorough  a  test  development  process  as  that 
for  algebra  courses.    But 'we  had  learned  from  the  earlier  item  review  procedures 

„  and  pilot  testing,  and  so  we  were  able  to  develop  and  validate  tests  for  ore- 

calculus  .courses  within  the  fall  academic  term  of  1975.    Again,  posttests  were' 

I  , 

-  ready  in  advance  of  the  demonstration.    It  was  the  start  of  the  1976  calendar  / 
year  when  (^colleges  offered  multiple  sections  of  pre-calculus  cour*ses  on  the  ' 


' TICCIT  system. 


|- -        Besides  the  posttests  fo^  algebra  and  prV-calcukv* courses,  our  evaluation 
Jailed  flir  measures  of^stu^iffc^chievement  that 'would  tap  immediate  learning 
and  retention.    But  post test  preparation  took  precedence  over  these  measures. 
-Developing  tests  on  -particular  topics,  which  were  to  be  measures  of  immediate 
learning,  began  after  our^Ktial  versions  of  the  posttests  had  been  tried  it  '. 
the  colleges.    Still  it  was  possible  to  develop  and  pilot  topical  tests  early 

j  in  the  academy  year  after  posttest  field  trials:    it  was  the  fall  temyjf 
1974  when  the  colleges  first  administered  the  topical  tests  on  linear  equations 

■  -  ■  ■  3  ■  "  ■  v 

and  algebraic  expressions.    The  construction  of  these  tests  was,  by  then,  a 
straightforward  procedure.    We  had  an  extensive* pool  of/ test  items  that  covered 


bo tf  topics,  a*  had  already  bean  through  teat  reviewa  by  fa<*lty  and  auttajrs. 

The  ease  Item  pool  and  experience  eased  the  preparation  of  tests  for  retention. 

%ese  were  to  be  administered  as  course-specific  pretests  in  the  academic  term 

subaequent  to  student  completion  of  a  target  courae.    They  were  also  the  lae* 

math  achievement  tests  in  terms  of  our  priorities  for  preparation  an<%he 

schedule  for  test  construction.    Thus  it  was  the  demonatjralAoTiSjeriod  of  the 

TICCIT  project,  1975-76,  before  we  reached  this  stage  in  test  parffcration. 

Perhaps  it  seems  that  the  preceding  account  has  omitted  an  essential" 

component  of  the  achievement  testing  in  math  courses.    It  has  not  touched  on 

«etests  as  controls  for  students'  entra*a*J»kllls.    This  is  because  /the  general 

Jpre^jtat,  differed  from  the  other  measures- of  achievement  in  an  important  respect: 

V^aa  a  standardized  testf^-  We  had  considered  the  use  of  special  pretests  but 

retefef"  them.,   Although  the  same  pretest  as  poet  test  might,  enable  us^to  compare 

curricufa^In^erme  of  achievement  gains;  there  would* sV  dangers  to  such  a 

testing  scheme.     It^ght  simply  alert  students  to  what  final  exams  would  cover 

and  thus  pose  a  test-reactive  threat  to  our  study's  validity  (see  Campbell  & 

^Stanley,  1963).     It  might  also  be  inappropriate  to  make  curricular  comparisons 

on  the  basis  of  change  scores  from  pretest  to  posttest  (see,  for  example,^ 

Cronbach  &  Furby/  1970)."   On  a  more  basic  level  itj»de  little  sense^to  ask 

students  questions  about  mjfcerial  they  had  not  been  te%ht.    Th^n  we  would 

expecjt  consistent,  low  scores.  uWhat  we  sought  instead  was  a  general  test  of 

students'  background  and  preparation  for  math  courses.     Such  a  pretest  would 

'  reflect  differences  in"  students' ^fl*^nice  ability  and,  presumably,  account  for 

< differences  in  posttest  performance  that  o.ther  variables  might  not  explain. 
\  * 
We  chose  as  our  pretest  a  part  of  the  Comparative  Guidance.and  Placement 

r 

Program  (Educational  Testing  Service,  197,3).    Tb^s  program  offers  a" wide  range 
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of  service*  t9  cwnmunlty  colleges.    It  .has  a  biographical  inventory,  a  compara- 
tive interest  index,  and  eight  tests  especially" designed  to  help  counselors  or 
faculty  advise  students.    One  college  involved  in  the  TICCIT  project  had  earlier 
participated  in  the  progr«i  and  the  other  was  receptive  to  use. of  the  program's  ' 
a«th  tests.    We  recommended  Part  1  and  Part  2  from  Level  D  of  the  CGP  math  tests. 
Each  of  these  two  sections  takes  about  twenty  minutes  to  complete  and  has 
thirty-five  questions.    The  first  partners  computational  skills:  arithmetic 
operations  with  whole  numbers,  fractions,  numbers  with" decimal  values,  andN 
percentages.    The  sec^d  places  emphasis  on  relations  *preen  quantities  (i.e.,  # 
equal  ^fT  greater  thM,  less  their),  but  students  must  work  with  Unknown  terms 
t    from&near  equations,  exponents,  quadratic  expressions,  thd  coordinate  planes 

in  order  to  answer  thejjgpions.  •  This  Second  section  J*  a  test  of  student**  W 
'  readiness  in  elemenrtn^algebra. 
....  The  standardized  pretest  was  adonis  teriTd  '3fc  a  triaifeasis  in  the  second 

*  half  of  the  1973-74  academic  year.    Scores  on  the  lest  appeared  to  reflect 
%  f  Aif eren«e*  «nong  students  as  well  as  differences' across  courses.    At  PhoWii 
'%W  ^  Pretest  scores  were  35.11,  46.26,  and  53481  across  the|r  ^trodud€ry , 
and,; college  algebra  coura*  sequence.    The  standard  deviation  \ 
withiti^h  course  fell  close  to  ten.poi^ts,  base^on  sample  si'^es  of  267,  253 
|    and  125  studentsAfprjbe  three  |«u^Wgec^vely .    Thus  the  CGP  pretest  was- 

sensitive  to  student  differences,  Wj^Tg'khe  size  of  the  standard  deviation, 
"     and.ta  differences  in  students'  iakth>bk|k|rpund,  ^^S&d  by  the  jump  iff 
the  average  score  through  the  algebra  sequence.    DalHollected  at  Alexandria 
confirmed  this  evidence,  for  the  CGJP  test's  appropriateness  in  the  evaluation. 
There  we  tested  students  at  the  start  of  three  distinct  course  sequences: 
fc^Aljsbra  I;  General  College  Mathematics  I;  ,gri  College  Mathematics  I.  As  expected,,;. 


J* 


the  pretest  mean,  was  fewest  for  the  least  stringent  sequence  in  terms  of  pre- 
requifit*  high  school  math*aa*ics  and  highest  for  the  sequence  thaWequired 
sJrSst  high  school  background.    In  Algebra  I  the  mean  was  34.67<while  in 
Silage  Mathematics  I  it  was  48. 9f.  ,  Again  the  standard  deviation*  ware  in  the 
neighborhood  of  ten  points.    Anv estimajs  of  pretest  r*||M>Uity  gsve  an  internal 
||onsistency  index  of  .92.    This  estimate  was  drawn  from  863  pretests  taken  in 
the  fall  semester  of  1974  at  Phoeni*^  It  is^ comparab le  to  the  reliability 
estimate*  reported  in  the  CGP  technical  manual,  .89  (Educational  Testing  Service, 
1973).    We  were  well-satisfied  that  the  CGP  provided  a  ™U-^Pd  reliable 
pretest  for  o^p evaluation,    n  *  j| 

The  preparation  of  posttests  followed  from  the  (Pelopment  of  an  extensive 
item  pool.    The  first  courses  scheduled  on  the  TJ^ClT  system  corresponded  jo. 
units  14  through  23  of  the  math  courseware  (see  Figure  1.3  in  Chapter  1  or 
Appendix  A).    So  our  examiners  developed  two  or  three  test  questions  f6r  each 
segment  in  the^e  units.    The  item  pool  consisted^  over  250  test  questions- 
There  were  mult^le^nhoice  as  well  as  open-ehdeV^estions.^  We  had  decided,  that 
a  multiple-choice  format,  despite  its  concomitant  ease, of  scoring  and  processing^ 
piaced  an  unnatural  constraint  on  student  responses  in  final,  exams.  ,  Math 
instructors  \ended  to  use  open-ended  problems  for  their  final  exams.  anTurged  us 
U>  do  likewise  in  the  posttests.    Of (course  we  wanted  our  posttests  to  be  , 
^  adopted  as  departmental  final  exams  (in  the  belief  that  tfcis  would  constitute  a 
"jstrong  argument  for  their  content  validity)  and  therefore  offered  a  compromise 
solution  with  both  formats,  . 

Instructors  at  the  colleges  revieired  the  item  pool  in  blocks  of  questions 
grouped  according  to  their  courses.    While  the  ten  courseware,  units  represented 

*  '  -   ■  ■  a 

-IceiT's  algebra  course,  they  in  fact  corresponded  to  five  different  courses  at 
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th.  community  colleges.    There  was  a  group  of  potential  teat  items  for  each  of 
the  five  courses,  with  considerable  overlap  among  the  groups.    It  is  easiest  to 
,0  1Uu8tra^  this  point:    Figure  2.1  represents  the  match  between  TBfclT  units  and 
coilege  courses  as  proposed  early  in  the  project.    Units  14  through  22  account 
for  nearly  all  the  material  in  Math  007,  I06,  and  108  at  Phoenix  and  Math  31  and 
181  at  Alexandria.     Instructors  told  as  whether  an  item  was  appropriate  for  one 
of  these  courses,  needed  r|gpton  before  further  consideration,  or.  was  unacceptT 
able.    From  their  item  reviews  and  suggestions  for  addt^j^^tix,^  1$jm&-:'  S 
regular  cla ss  coverage  extended  beyond  that  of  TICCIT,  courseware) ,  we  developed 
the  first  versions  of  poattests. 

Thesf^rst  posttest  forms  we$e  not  th^last.     In  genera^  there  were  three^ 
versions  of  each  test,  and  four  versions  were  required  in  Wated  cases  of 
repeated  curriculum  changes.    W|made  these  revisions  in  order  to  accommodate 
instructors'  recommendations  for  fair  tests  as  well  as  to  improve  the  tests  with 
the  benefit  of  item  data.    For^example,  one  course  stressed  .factoring  and  inatruc- 
tors  pointed  out  that "our  posttest  was  inadequate  in  its  emphasis  on  this  aspecipj 
of  the  course.  So  we  adjusted  the^  number  of  test  items  to  rS'iect  course  emphasis 
as  well  as  coverage.    There  were  also  some  questions, deleted  from  the  posttest. 
Particularly  those  items  on  arithmetic  operationa  with  positive  and  negative 
whole  numbers,  such  as  14  ♦  (-2)  or  3(25-9(2)]  or  2(-6),  proved  to  be  too 
elementary.    Nearly  all  students  gave  correct  answers  toS^  questions.    We  also 
found  content  low  item  diff  icult^foj^ questions  about  simple  set-subset  ' 
relational  and  therefore  deleted  these  questions  as  well.  *' 

To  il^strat^h^type  of  items  on  the  posttest  two  pWemsHr™  an  inters 
mediate  algebraflHBj^  ln  Flgure  2.2.    ^  multlple  ^c  problem  aUo 

^serves  as  an  It  kind  of  revisions  were  made  to  improve  individual 
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Figure  2.1 


test  item**    Originally  that  problem  called  for  an  ordered  pair  (x,y)  as  a 
solu^n  to  the  two  linear  e|uations.    But  with  alternative  pairs  given  as 
response  choices',  a  student  might  solve  the  problem  by  simple  substitution.  The 
revision  discesjgps  this  method  of  solution.    Tbe  other  problem  represent. ^the 
type  of  open-ended -questions  used  In  the  posttests.    This  particular  item  was  an 
addition  made  ftp  reflect  course  emphasis. 

Field  trials  fot  all  tests  except  those  on  retention  were  held  in  con- 
junction,.^ the  first  use  of  the  TICCIT  system  for  teaching  algebra  courses. 
But  these  .(rials  had  been  preceded  by  test  administrations  focus*!  on  refining 
items.    So  the  197A-7*  test  forms  already  incorporated^revisions  based  on 
earlier  data  collection.    The  length  ef^sttest.  then  ranged  between  54  and  64 
separate  items,  witheach  mUltfplelpart  problem  counted  as  several  items.  The 
two  topical  tests,  one  on  linear  equations  and  another  on- algebraic  expressions, 
had  25  and  26  items  respectively .    Internal  consistency  estimates  of  posttest 
reliability,  came  to  an  average  of  .84  in  the  fall  academic  term,   '^fter  „e  made 
some  minor  changes  and  shortened  forms  by  about  three  items  per  tett,'  the  .  ■ 

average  in  the  spring  term  was  .89.    Reliability  of  the  topical  tests  averaged 
.88  for  the  same  academic  year.  ^  * 

Test  reliabilities  fo'V  the  19^5-76  academic  year  appear  in  Tables  2.1  and 
2.2.    This  period  was  <che  demonstration  phase  of  the  TICCIT  project  for  community 
college  math.     The  figures  in  these  tables  apply  to  the  data  reposed  later  (see 
Chapter  5:     Student  Achievement)  as  part  of  our  evaluation  ot  the  impact  of  the 
TICCIT  program  on  student*  achievement.  ♦  . 

As  expected,  the  standardized  pretest,  identified  as  test  form  601  in  T&le 
2.1,  has  a  high  reliability.    The  course-specif ic  pretests,  forms  622,  623  an'd 
624.  were  constructed  to  meet  our  dual  need  for  a  measure  of  retention  from  a 


Examples  of  Problems  from  Math  Poattests* 

MuXfciple-choice  Format: 

1?.    M  2y-A  aid  xfy-6,  theri^  x-y- 

(A)    -6         (B)    -2         (C)    2         (D).#  6 

Open-ended  Format :     *  - 

44/    Factor  completely  each  of  the  following* 

(A)    x2-25y2  *  %  >'. 

(J);  2x2-x-6 

(C)  5-20x*  d> 

(D)  49x^-25^ 


W.gure'2.2 
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•   <Pretest  Reliability:    Intersil  Consistency  Estimates 

Alexandria  and  Phoenix 
Mathematics  Couf&es  \ 
Calendar- ¥ear  1976 


Academic  Term 

Course 

Test 
Form 

Number  of 
Students 

Total 
Number 
of  Items  ■ 

_r 

Fall  Semester 

Math  007 
Math  106 
Math  108 

601 
601 
601 

422 
180 
133 

70 
70 
70 

.92 
.92 
.92 

Spring  Semester 

Math  007 
Math  106 
Math  108 
Math  117 

601 
622 
601 
623 

332 
229 
153 
150 

70 
20 
70 
20 

-.91 
.70 

..91 
.77 

Fall  Quarter 

Math  31 

601 

221 

70 

.90 

Winter  Quarter 

Math  32 

624 

44 

20 

.72 

Spring  Quarter 

Math  31 
Math  32 

601 

624 

130 
75 

70 
20 

•  .92 
.66 

Averages : 

601 

622,  623,  and 

624 

70 
20 

.91 
.71 

ERIC 
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Posttest  Reliability:  ■  Internal  Consistency  Estimates 
Alexandria  and  Ptfoenix 
f!athetnatics  Courses 
/  "  Academic  Year  1975-76 


Posttests : 


*Tefct    -  Number  pf 


.   Academic  Term  •'. 

Course  • 

Form 

Students 

Fall  Semester 

Math 

007 

6L4 

182 

Math 

106 

618 

101  • 

Math 

108 

.'619 

54 

Spring  Semester. 

Math 

007 

614 

181 
'  108* 

Math 

106 

615 

Math 

108 

616 

'55  •. 

Math 

117 

626 

71 

Fall  Quarter 

Math 

31 

609 

122C105)1 

Winter  Quarter 

Math 

31  - 

609 

113(80). 

Spring  Quarter 

Math 

31 

609 

79(41)1 

Math 

•  32 

.  627 

922 

Averages 


Total  V  ■ 

Items 

m  uiiiu  ci 

for  Common 

of  Items 

,  r_ 

Oblectives 

r 

57 

.92 

48 

.91  » 

•  49 

.85 

.  43 

.84 

JO 

.88 

50 

.87 

57 

.86 

48 

.86 

55  - 

'  .86 

48 

.85 

.61  . 

.85 

55 

.85, 

46 

•  90 

38' 

.90 

■i 

54 

.90 

44 

.90  i 

54 

.91 

44 

.92  , 

54 

.94 

44 

.93 

41 

.86 

34 

.  85  - 

s-/  53 

.88 

45  ' 

.88. 

Topical  tests: 

Linear  Equations  - 

Fall  Quarter^ 
Winter  Quarter 
Spring  Quarter 
Spring  Semester 

\     Algebraic  -Expressions  - 

Spring  Quarter 
Spring  Semester 


.'> 

s. 

i. 

) 

.  ■<  * 

.81 

Math  31 

617 

77 

4  22.. 

Math  31 

617 

-77 

22 

.84 

Math  31 

617  , 

'57 

'22 

.82  , 

Math  007 

607, 

80 

26 

.80  « 

r 

( 

*  V 

Math  32 

608 

55 

25 

.76 

Math  108 

608 

49 

25 

.85 

Averages:  24 


,81 


s 


Hftiile  the  reliability  estimate  for  total  score  is.  based  on  all  posttests,  that  for^ 
oblectives  common  to  TICCIT  and, lecture  conditions  excludes  posttests  taken  in  programed 
instruction  classes.     Thus  there  is  a  lower  number  of  students  for  these  estimates  oY 
consistency  on  Math  31  common  objectives. 

r 

Estimates  for  Math  32  include  data  from  both  the  winter  and  spring  quarters.  The 
number  of  students  completing  this  course  in  the  winter  prevented  separate^analyses 
for  each  term.  ,  R 
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4 

prior  course  as  ,  jae  1 1  as  a  measure  of  student ^preparation  for  the  next  course  i& 

i  -  "  /  ;       '  - "  •  J 

Algebra  sequenced  «  For  both  purposes  a  tes»t  of  what  a  student  had  ^earned  in 
the  prerequisite  course  seemed  reasonable,  /nd  one  test  instead  of  J^wo  was 

•  i        .  . 

always  desirable  so  as  'to /reduce  the  evaluation' s  interruption  of  teaching 

activities.     The  avetfa^e  reliability  of  these  course-specific  pretexts  falls 

below  that  of  the  CGP  Math  D  test.     However,  test  length  perhaps  explains  t^is 

difference.  ?  there  were  only  twenty  items  in  each  course-specific  pretest. < 

Table  2.2  presents-  reliability  estimates  for  all  pdsttests  and  topical^^a^- 

tests  used  Co  isse$l&  student  achievement/   The  high  postfest  celiabilities 

seem  to  reflect  phe  care  taken  in  their  preparation.     The  actual  analyses  o£/ 
;  -  ^  •  .  „*  •    ( •  '  '  '  >J  •       * '  ' 

posttest  results  rely  .on  those,,  items  for  objectives  shared  by  the  classroom  and 

TICCIT  conditions.^    So  the  table  also  ^feports  reliabilities^ for  each  posttest 

based  only  on  these  common  objectives.     There  is  hardly  a  difference  between 

"       \  *  -J  S  ■   '  •■'  * 

test  reliability  for  total>  score  and  that  for  commo>K  subscore,  probably  because 

at  least  four  out  of  every;  five  items  tested  a,  common'  objective,    for  the 

topical  t^te  the  average  reliability  was  satisfactory  iespite  the  low  number  of 

"    /  f  ft  '  .  'r      ,     .       ^  \f 

items.     The  discrepancy  between  cour s§-specif ic  prefc/st  reliability  and  this 
average  for  topical  test  might' bie  attri^uted^to  the/breadth  of  test  coverage. 
While  protests  spanned  an  entire  term'iSf  \materihl -in  just  twenty  items,  topical* 

\  ^       "  ,  f  •      a      '  '  \ 

tests  focufeed, on  one  particular. component  of  a  course dand  thus  covered  about 

y  '  ;  ,*  *  /  .  *  -      s  >  .  ^  .  *  .  •  * 

one-fifth/as  much  material  with  an  e^uiva Lent  number  of  it<^n?^     Their  limited 

content  coverage  certainly  contfributed  to  the  jtOnsistency  o'f  topical  tests  as  ! 

achievement  measures.'    ,  ^  N 

The  two  reliabilities  for  each  posttest,  one  for  total  score  and  arfbther 

for  the  subscore  based  on  common  objectives,  suggest  the  importance  of  item 

■  J      .   r  .       '  .'  . 

classification  and  perhaps  raise  the  issue  of  test  Validity «     It  was  the^nath' 


/ 
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faculty  at  the  demonstration  sites. and  the  courseware  authors  who  determined 
\  whether  an  item  was  related  to  their  instructional 'objectives.  We,  simply  tpok 
•both  sets  of  reviews  and  established  four  categories  for  each  course' s  objectives: 
(1)  objective  cp^n  to  both  lecture  and' TJGCIT  coverage;  fo.  objective  unique 
to  the  curriculum  in  Regular  classes;  (3)  objective  unique  to '.tHiPUJCCIT  course- 
ware; and  (4)  objective  beyond  the  scope  of  the  instructional  programs^  This 
classification  scheme  for  objectives  helped  us  to 'avoid  measures  of  resllts  with 
undue  emphasis  on  one 'curriculum  or  the  other.  /Ind  our  analyses  of  achievement' 
results  did  use  common  scores  as  the  basis  for  comparisons  between  program 

outcomes.  ^  . 

r  Even  with  comparisons  confined  to  objectives  shared  between  programs,  it  , 

JLs  possible  that  the  TICCIt  prog ram* would  lead  to  patterns  of^ math  achievement  > 
different  from  those  of  .lecture  classes.     So  we  classified  each  test  item  by 
ability  level  and  content  category* as  well  as  objective.    This  classif icati/n 
scheme  made  it  easier  to  detect  finer,  albeit  less  reliable,  differences  in 
program  outcomes.     Such  detail  was  consistent  with  our  intent  to  doaument 
TICCIT'.s  strengths  and  weaknesses ,  and  to  . present  evaluative,  inf omation^ief  ul 
in  making  decisions  about  program  adoption  or  refinement. 

'     The  categories  for  classifying  posttest  items  appear  in  Figure  2.3.  The 
ability  levels  closely  follow  £hose  used  for  standardized  testing  programs,  for 
example  the  Sequential  Testsfof  Mutational  Progress  (Educational  Testing 
Service,  1971).     A  question  for  factual  recall  typically  called  for  use  of  a 
definition  (e.g.,  "Which  o£  th|  following  is  a  natural  numbVrT^or  "List  the  t 
first  five  elements  in  the  set^of  even  integers  greater  than  0."),  Manipulation^ 

Commonly  involved  ariVhyfe tic  computation  (e. g.  ,  "In  its  simplest  toflQ^.  -  .  

or  "Work^each  of  the  fallowing  and  write  the  result  Jas  a  rational  number  in  its 


J,    .  ■  ^ 

.  > 


/  <:  ')  ■  "  ■   '  c 
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*  Item  Classification  r< 

for  Mathematics  Posttests 


Categories:  '  .  * 

Object ives\ 

.1  -.objective  (common  to  both  lecture  and  TICCIT  coverage 
'       "2  -  objective  unique  |to  curriculum  in  regular  classes 

3  -  pbjective  unique  to  TICCIT  courseware 

4  -  objective  beyond  the  scope*  of  the^/in^tructional  programs 
-  Ability  Level 

^0  -  factual  recall  -  t 

,       I  1  -  manipulation 

•  -  <V 

2  -  solution  of  routine  problems- 

3  -  demonstration  of  concept  comprehension  '  / 

Content  Category. 

1  -  arithmetic 
.  2  -  algebra  * 

k 

0a>-  simplifying<lincludir>g,  properties  of  ai}d  computations  with  signed 

numbers  and  complex  numbers  - 

)  - 

(b)     equations,  including  substitutions,  equivalent  equations,  solution? 
of  equations,  and  word  problems  \  i 

'     '  :  -.\ 

\(c)     exponents,  radicals,  and  logarithms 

\  ^ 

(d)  factoring,  multiplying,  and  dividing  algebraic  expressions  including * 
quadratics 

(e)  number  line  .and  coordiante  plane 

(f)  inequalities ,  absolute  value,  ajid  sets  including  Solution  sets 

(g)  sequences,  arithmetic  and  geometric  progressions 
^               (h)  permutations  and' combinations ,  binomial  theorem 


Figure  2,3 


simpLest  form:  (A)  |  +  ±  ,  (B)  ^|  x  f  ,  (C)  10  +  |  .").  The  open-ended  problem 
given  in  Figure  2.2  illustrates  the  kind  or  item  considered  as  belonging  within 
the  level  for  solution  of  routine  problems.     To  be  classified  as  an  item  that 

tapped  demonstration  of  comprehension  of  a  concept,  a  question  had  to  go  beyond 

f 

the  bounds  of  the  routine  problems  encountered  in  a  course.     Such  problems 

required  that  students  apply  tffa^l/tar  rules  in  an  unfamiliar  context.  For 

Sample,  although  students  ha^yKrked  with  word  problems  in  beginning  algebra 

courses,   these  problem^w^enot  as  straightforward  as  strictly  numerical 

systems  of  linear  equations.     So  word  problems  such  as 

Tanks  P  Ind  Q  tlgether  hoJ.d  630  gallons  of  gasoline.     If  tank  P  - 
holds  154  gallons  more  than  tank  Q,  how- many  gallons  does   tank  Q 
hold? 

and  other  problems   that  involved  combinatiari'3  -of  rules   (e.g.,   "If  2     x  (2  )     =  2  , 

then  x  =   _\^^nd  there^o.re  presented  otherwise  routine  concepts  injnj^a^ai  liar 

situations,  came  unfcer  the  ability   level  for  dAipnsafcafrion'of  concep  t\comprehens  ion  v 

The  examiners^  responsible  felt  preparing  posttests  establish^  the  content; 
categories  from  inspection  of   the  posttest  items.     As  might  ^e  expected  these 
content  categories  conform  fairly  well  to  the  /courseware  units  and  textbook  \ 
Chapters.     And  since  they  correspond  to  traditional* topics   the  scheme  is  probably 
se If -exp lanatory . 

The  result  of   the  three  dimensions  for  item  classi f ica-tion  is  a  posttest 
breakdown  such  as  that  depicted  in  Figure  2/4.  \  This  gives  the  number  of  test 
items  within  each  cell  of   the  classification'  $che^%    %,t  should  be  apparent  that 
too  few  items  fall  within  a  cell  for  the  purposeW  making  comparisons  be.treeri 

'    '  ;  \     \         •  ■      *  *  * 

p'ro^rams.     Instead  we   relied  on  tlte '  marginal  \§iart&  When  looking  at  results  , 
across  programs.     As  with  the  analyses  of'overall  achievement  effects,  only  those 
Ji  terns  on  common  objectives  went   into  the  subscores  for  the  fou*  ability   levels  and 


1. 

2a. 
2b, 
2c. 
2d. 
2e. 
2f. 

2g. 
Zh. 


ArythmetfLc  .  . 
Simplifying  . 
Equations  . . . 

1  'J 

Exponents  . . . 
Factoiling^^^ 
Number  line  . 
Inequalities  , 
Sequences  .  .  . 


Permutatidhs  . . 


f 


Example  of  item  Coverage 
for  Math  Posttests 
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4.     Objectives  beyond  programs 


Objectives  unique  to  TICCIT 


V 

2,     Objectives  unique  to  lecture 
1/  .Objectives  common  between  programs 
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Ability  Levels 


7     Number  of  items  from  beg^rfning  algebra  posttest/  that  fall  within  each 

/  (  !  - 


cell. 


Figure  2.4 
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V  j  _      r)f  course  not  aft  subscores  existed  for  each  postte^t. 

the  nine  content  categories.     Ut  course  not  an  ^ 

i     v  -  nn  1  fPTTi*;  nn  qpbuences  or  permutations,  while  that  for 

The  beginning  algebra  test  had  no  items  on  sequenuca  f 

advanced  algebra  had  none  on  arithmetic.     Complete  item  classifications  for  the  post- 
tests  appear  in  Appendix  C.     These  should  both  demonstrate  postteat  validity  through 
matches  with' the  course  coverage  and  convey  some  idea  of  the  applicability  of  our 
results  to  other  community  colleges'  courses. 

Perhaps  the  best  evidence  of  validity  for  our  math  achievement  tests  comes 
from  the  manner  in  which  the  colleges  used  them.     The  math  department  at  Phoenix 
adopted  them  as  their  final  exams  and  determined  students'  grades,  in  part,  by 

ThP  aleebra  courses  at  Alexandria  had  simple 
their  score  on  our  posttests.     ine  aLgeDra  cour&ca 

h 

pass-fail  grades  dependent  on  whether  a  student  finished  a  course  and  how  much 
progress,!"  had  made  in  the  course.     Our  posttests,  then,  4^  not  contribute  to  . 
student  grades  at  Alexandria.     But  when  a  major  curriculum  change  forced  the  . 
college  to  try  the  TICCIT  program  for  another,  comparable  course  sequence,  the 

nur-  nnciffecifc;  as  achievement  measures  in  their  own 
math  department  chose  to  use  our  posttests  as  aunc 

study  of  student  performance*  y 

Achievement  Tests:     English.     The  process  of  test  preparation  for  courses 
in  English  composition  was  quite  different  from  that  for  math  courses.  There 
is,  after  all,  a  marked  contrast  in  the  subject  matter.-    Although  an  item 
classification  scheme  might  reveal  much  about  the  cur  richer  emphasis  of  a  math 
course,  it  te^us  little  about  composition  cour.ses  where  the  primary  concern  is 
st-dent  writing.     We  took  epay  samples  ^from  students  as  the  most  direct  measures 
oAchievement  in  composition  courses.     Since  certain  types  of  objective  items 
I    show  a  strong  relationship  to  writing  ability,  we  also  developed  tests  of 
composition  skills  in^a  multiple-choice  format.  ( 
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The  TICCIT  system  first  supported  courses  in  English  composition  in 
the  summer  term  of  l975^^\lmost  three  years  earlier  we  had  reviewed  Initial 
courseware  specifications.     It  was  so  difficult  to,  transfer  a  traditional 
and  sound  English  curriculum  to  a  mainline  format  that  there  were  seritms^  /  . 
schedule  slippages  in  courseware  production  and  significant  changes  to  the 
•  content  specifications.     The  developers  of  the  TICCIT  program  also  went  through 
unexpected  professional  disputes  among  their  composition  authors  and  with 
college  instructors  in  attempting  to  define  an  optimal  course  on  writing.  Of 
course  this  affected  our  schedule  for  instrument  preparation. 

If  we  had  waited  until  the  TICCIT  ptogram  supported  composition  classes, 
the  time  span  before  the  demonstration  phase  would  have  been  too  brief  for 
instrument  development  and  field  trials.     We  began  preparing  appropriate  tests*  in 
.the^winter  of  1974.     From  visits  to  the  colleges  in  the  preceding  year  it  had 
become  clear  that  only  an  essay  sample  would  lead  to  a  consensus  among  participants 
.  on  testing.     Examiners  with  backgrounds  in  English  test  development  met  with 
instructors  and  encouraged  them  to  submit  suggestions  for  appropriate  essay 
topics. 

Nearly  fifty  recommendations  were  sent  to  us  from  the  colleges.  From 
these  we  selected  ten  as  possible  topics  that  would  be  familiar  enough  to  all  * 
students  for  them  to  write  under  time  constraints.     A  list  of  these  topics, 
slightly  revised  so  as  to'  conform  to  a  testing  situation,  was  then  returned 
to  the  colleges.     Toward  the  end  of  the  spring  term  of  the  1973-74  academic 
year  instructors  cooperated  with  the  evaluation  by  collecting  student  responses 
to  the  topics.     These  writing  samples  constituted  the  data  base  which  determined 
the  pretest  and  posttest  essay  topics.     Of  course  data  or  writing  samples 
themselves  do  not  determine  topics.     Rather  instructors  scored  the  samples  at 
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sessions  conducted  by  oUr  examiners  and  then  we  chose  topics  for  certain 

r 

characteristics. 

'    But  what  criteria  do  ln8troCfor«»  consider  ln  a8se3sing  the  worth  of 
a  student's  writing?     Thls  queStion  was  pertinent  to  understanding  scores 
assigned   to  essays  and    to  devf;loping  objective  tests  of  writing  skills.  So 
prior  to  holding  scoring  sessions  and  discussing  multiple-choice  test  formats, 
we  distributed  a  very  sh0rt  questi°nnaire  to  Members  of  the  English  faculty  at 
participating  colleges.     In   ^  the  questionnaire  had  just  one  question:  "How 
do  you  assess  the  worth  of   &  stUdent's  writing?     Fof  which  of   the  following  do 
you  tend   to  give  credit?-     The.facuUy  response  Was  remarkable:     within  three  ' 
weeks  85%  of  the  instructor  had  returned  forms. 

A  summary  of  their  responSeS  appears  in  Table  2.3.     Quite  obviously  logical 
organization  and  quality  of   thoUght  represented   important  considerations  in 
assessing  student  writing>     ^^ima  te  ly  half  of  the  instructors   further  t 
indicated  that  these   two  aspectS  of  writing   took  precedence  over  other  consid- 
erations  in  judging  student  essays.  '  Ci°se  behind   in  importance  were  correctness 
of  expression  and  unity.      ^  ^  lnteresting   to  note  that,  despite  differences  in 
teaching  composition  skllu>  there  was  a  consistency  in  judging  student  writing 
both  within  a  coUeg*  and  betWeen"  colleges .     Perhaps  this  consistency  and  the 
distribution  of  .respond  various  aspects  q£  writing  will-  ease  others- 

concerns  about  what  criter.a  £aCuUy  aPPlied   in  Scoring  our  essay  pretests  and 
posttests. 

At   the  sessions  held   in  che 'spring  of   l97^    .nstruCtors  assigned  scores  on 
a  scale  of  one  to   four   Points,    Such  a  holistlc  scale  may  seem  too  narrow  in 
range  for  assessing  student  writing-     Actually %   thoUgh,  out  purpose  was  evaluating 
a  curricular  program.     ^  ^  ^  of  tour  PoSsible  choices  contributed  to  the 
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%       '  Aspects  of  Writing; 
Faculty  Considerations  in  Rating  Student  Essays  * 


Alexandria         .  Phoenix  Total 


Frequency 

Percent 

Frequency 

Percent 

Frequency 

Percent 

Number  of  respondents 

15 

100 

19 

100 

34 

100 

Aspects  of  Writing 

• 

Sophistication  of  vocabulary 

8 

53 

n  1 

58 

19 

56 

•Originality  of  expression 

12 

80  /\ 

12 

63 

24 

71 

Expression  of  writer's  personality 

6 

/ 

40 

? 

32  '  . 

/  -13 

38 

Quality  of  thought 

.15 

100 

19 

100 

34 

100 

Sophistication  of  sentence  structure 

13 

87 

15 

79 

82 

i 

Awareness  of  situation  and  audience 

11 

73 

15 

79 

26  i 

76  i 

Correctness  of  expression 

15 

100 

18 

95 

33 

97 

Logical  organization 

"  15 

100 

19 

100 

34 

loo 

Expression  of  feeling 

8 

1 

9 

53 

7 

*  32 

-  15 

44 

Sincerity  of  expression 

60 

12 

63 

21 

62 

Economy 

10 

67" 

13 

68 

23 

67  ' 

Unity 

,  14 

93 

18 

95 

32 

94 

uiner 

♦ 

thesis  statement 

3 

16 

1  1 

support  of  theme 

3 

16  _ 

Table  2,3 

* 
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reliability  of  essay  scores.     A. wider  spread  of  possible  scores  would  make  it 
B&re  difficult  to  *rea.ch  a  consensus  on  the  appropriate  score  for  sample  essays, 
a  necessary  preliminary  step  in  the  scoring  session?.     Several  hundreds  of 
.  writing  samples  were  scored,  in  the  pilot  testing  for  essay  topics. 

From  the  topics  tried  at  the  colleges  we  selected  one  as  a  pretest  and  " 
another  as  a;posttest.  .  These  selections  were  based  on  the  distribution  if. 
of  scores,  the  frequency  "of  score  discrepancies,  and  comments  made  by  the 
instructors  about  student  responses.     We  wanted  an  essay  topic  familiar  enough 
to  students  for  them  to  be  comfortable- in. Writing  an  essay, within  twenty  minutes. 
Yet  we  also  wanted  topics  that  would  reflect  differences  among  students.     So  we  ( 
looked  for  a  spread  in  the  distribution  of. scores.     Since  two  instructors  road^ 
each  essay,  it  was  possible  that  a  topic  might  be  particularly  susceptible  to 
'    divergent  scores.     Such  a  tojiic  would  not  be  suitable  for  the  evaluation. 

Therefore  we  chose  topics  with  a  minimum  number  of  score  discrepancies.  Finally, 
instructors  contributed  to  our  decision  about  essay ^ topics  through  their  comments 
about  students'  reactions  to  the  task  in  classes  as  well  as  the  quality  of  their 
students'  papers. 

The  essay  topics  for  pretesting  and  posttesting  appear  in  Appendix  D.  -i 
These  were  subject  to  additional  field  trials  in  the  fall  term  of  the  1975-76 
academic  year.     Over  2700  students  responded  to  them  in  the  1976  calendar  year, 
the  demonstration  period  for  the  TICCIT  program  in  English  composition  courses. 
When  data  collection  came  to  a  close,  scoring  sessions  were  again  held  at  the 
colleges.  . 

The  reliability  of  the  essay  tests  really  depended  on  the  extent  of  agree-^ 
ment  between  independent  scorers.     Table  2.4  presents  the  correlations  between 
raters  for  the  essays  scored  at  Alexandria,  and  Table  2.5  for  those  scored  at 
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Essay  Test  Reliability:    Correlations  Between  Raters 
Phoenix  Scoring  Session 
English  Courses 
Calendar  Year  1976 


Pretest  Essay 


Raterl 


Posttest 
Essay 


N 
r 
N 
r 
N 
r 
N 
-  r 
N 
r 
N 
r 
N 
r 
N 
r 
N 
r 
N 
r 
N 
r 


B 
4 


13 
.61 
1 


14 


28 
,77 
19 

,66  .77 
19/  1 

./  - 


22 
.61 

.83 
0 


27 
.61 

13 
.10 
0 


C 
0 


46 
.74 
0 

15 
.66 
1 

20 
.79 

11 
.51 
7 


D 
1 


29 
.67 


E 
0 


25 
.49 

17 
.32 

11 
.79 

16 
.45 
9 


F 

G 

H 

I 

J 

K 

57 

20 

23 

13 

29 

33 

.79 

.49 

.76 

.88 

.77 

.67 

38 

0 

26 

26 

^30 

.63 

— 

.59 

.72 

.74 

.69 

'  12 

35 

24 

11 

43 

31 

.77 

.75 

.66 

.48 

.69 

.58 

40 

35 

53 

25 

19 

23 

.70 

.64 

.72 

.84 

.68 

.66 

12 

35 

29 

35 

20 

"24 

.61 

.68 

.73 

.68 

.70 

.67 

3 

0 

10 

18 

3 

.75 

.63 

0 

12 

13 

13 

2 

.  .71 

.56 

.52 

7 

12 

12 

5 

0 

.61 

.73 

0 

0 

2 

0 

0 

19 
.65 
8 


14 
.68 
0 


26 
.75 


12 
.72 


Averages:     pretest  correlation  .68 
posttest  correlation  .63 


Table  2.4 
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Averages:     pretest  correlation  .71 
posttest  correlation  .68 


Table  2.5 
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Phoenix.     Raters  (instructors  in  the  English  departments  at  the  respective 
"colleges)  were  unaware  of  where  essays  had  been  written  or  from  which  class, 
essays  had  come  since  all  identifying  information  had  been  masked  oh  the  student 
papers.     StUl,  it  was  possible  that  some  instructors  recognized  the  pretest  antl 
post  test,  topics  from  test  administrations  in  their  classes.     Indeed  it  did  seem 
that  some  instructors  employed  stringent  standards  for  the  highest  score  on 
posttests.     This  might  account  for  athe  slightly  lower  correlations  between 
raters  on  posttests  essays   than  on  pretest  essays.  ^ 

The  average  inter-rater  correlations ,  despite  a  range  from  .10  to  I.  00,- 
falL  within  reasonable  limits  for  this  type  of  judgment  reliability.     A  reii- 
ability  'of  .67  or  so  might  be  low  in  comparison  to  the  internal  consistency 
estimates  for  math  tests,   but  represents  a  satisfactory  level  consistent  with 
our  experience  in  essay  scoring.     We  had  taken  precautions  to  guard  essay 
reliability.     Whenever  raters  disagreed  by  two, or  three  points  in  scoring  a 
paper,   those  scores  were  discard||s|fcp  two  other  raters  read  the  paper.  The 
incidence  of  such  discrepancies,  was  very  low:     fewer  than  six  percent  of  the 
essays  had  to  be  reread.     And  the  second  pair  resolved  all  but  about  a  dozen  of 
the  discrepancies.     With  regard. to  the  consistency  of  scores  assigned  at  the  two 
sessions,   raters  at  Alexandria  and  Phoenix  went  through  the  same  sample  of 
student  papers  bef o^e^ac^ally  grading.     The  correlation  between  the  two  sets  of 
scores  was  .96  for  t*he  pretest  essay  and  .84  for  the  posttest  essay.  These 
coefficients  seem  to  justi.fyv.our  treatment  of  scores  as  if  assigned  by  one 
group  for  ail  essays. 

The  item  types  in  the  objective  tests  were  among  those  with  the  strongest 
relationship  to  the  criterion  of  actual  writing.     Godshaik,  Swineford,  and' 
Coffman  (1966)   reported  the  correlations  for  the  two  major  types  that  appear 
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(  iH^our  *  tests  as  .70  with  student  essays.     This  was  the  primary,  reason  behind  our 

decision"  to  use' these  item  formats,  for  there  had  to  be  evidence  of  a  relation- 

if 

ship  to  writing  if  instructors  were  to  accept  objective  tests  as  appropriate  for 

composition  courses. 

Figure  2.5  gives  examples  of  the  item  formats  from  the  objective  tests. 
The  first  type  requires  the  ability/to  recognize  writing  that  does  not  follow 
the  conventions  of  standard  written  English.     Such  items  accounted  for  over  60% 
of  the  objective  tests.  ^Another  fourth  of  the  tests  consisted  of  items  similar 
to  the  second  type  shown  in  iigure  2.5.     These  reduire  recognition  of  improper 
writing  as  well  as  sufficient  writing  skill  to  choose  a  clear  and  precise  manner 
for  rephrasing  sentences.     In  the  last  type  of  item  students  had  to  join  phrases 
together  and  thus  decide  about  relative  detail  and  logical  connectives  in 
sentences.     These,   too,  correlate  highly  with  actual  writing,  but  not  to  the  same 
extent  as  the  preceding  \^wo  types  (see  Godshalk,  Swineford,  &  Coffman,.  1966). 
Andvso  smch  items  amounted  to  just  one-eighth  of  the  objective  pretest  and 
posttest. 

The  item  format  may  tell  something  about  the  ability  tested,   like  the  item 
classifications^ for  math  posttests  according  to  ability  level.     It  omits  infor- 
mation comparable  to  that  given  by  the  content  categories  for  math  posttests. 
It  should  be  clear  that  our  foremost  concern  in  testing  was  assessing  students' 
writing.     Coverage  of  specific  grammatical  points  or  particular  structural 
conventions  was  secondary.     However,  a  breakdown  of  pretest  and  posttest  coverage 
may  be  found  in  Appendix  E.     At  the  item  level  it  is  apparent  that  the  objective 
tests  tap  traditional  points  in  writing:     subject-verb  agreement,  pronoun  reference 
adjective-adverb  confusion,  verb  form  and  other  skills. 
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Examples  of  Item  Types  from  Objective 
\^  Tests  of  Writing  Skills  4  >x 

Directions:    In  each  of  the  folldving  sentences  find  out  what  is  wrong,  ilf  anything.  In 
deciding  whether  there  is  something  wrong  with  a  sentence,  con8iderj_£ite^0^-/a  sentence 
should  be  written  in  standard  written  English*  the  kind  of  English  usually  found  in 
textbooks.    Remember  that  this  is  sometimes  different  from  the  kind  of  English  that 
*•  use  In  talking  with  our  friends.  * 

Some  sentences  are  acceptable  without  change.  ^ 
No  ^sentence  contains  more  than  one  error.  q 

If  the  sentence  has  an  error,  you  will  find  tnat  the  error  is  underlined  and  lettered. 
Assume  that^ll  other  parts  of  the  sentence  are  acceptable  and  cannot  be  changed. 

Wh*n  you  find  an  error,  select  the  one  underlined  part  that  must  be  changed  in  order  to 
make /the  sentence  acceptable,  and  blacken  the  corresponding  space  on,  the  answer  sheet. 

If  there  is  no  error,  mark  space  E.  . 

ANSWER 

*•    Mssy  of  his  admirers  believe  that  Vonnegut;  sees  life  in  the  same  way  Q[) 

A  B  ,    C  ' 

<  •  *  m 

as  them.    No  error  ^ 

d  e  m- 


8.    Washington  successfully  attacked  the  Hessians  at  Trenton  and  demonstrating 
to  the  British  that  the  Colonial  Army  was  still  strong.    No  error 


m 


0 
® 
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Figure  2.5(a) 


Examples  of  Item  Types,  from  Objective 
Tests  of  Writing  Skills 


Directions:     In  each  of  the  following  sentences  soijie  part  of  the  sentence  or  tfie  entfre 
sentence  is  underlined.    Beneath  each  sentence  you  vilfrfind  four  ways  of  writing  the 
underlined  part.    The  first  of  these  repeats  the .underlined  part  in  the,  original 
sentence,"  but  the  oWr'three  are  all  different.     If  you(think  the  original  sendee 
is,  better  than  any  of  the  suggested- changes yon  should  choos<f  answer  A^btherv^ 
you  sjjfcyld  mark  one  of  the  other  choices.     Select  the  best  answer  and  ^acken  the 
cor rajpond in g, space  on  the  answer  sheet, 


In  choosing  your  answers,  follow  the  requirements  of 'standard written  English,  the 
kind  of  English  usually  found  in  textbooks.     Re&mber  that  it  is  sometimes  different 
from  the  kind  of  English  we  use  in  talking  wittt  our  friends.     Pay  attention  to 
how  clearly  ideas  are  expressed,  whether  the  wlrds  convey  the  meaning  they  are 
supposed  to  convey,  and  how  the  sentence  is  constructed  and  punctuated.  Choose 
the  answer  that  produces  the  most  effective  sentence— clear  and  exact,  without  . 
awkwardness  or  ambiguity.     Do  not  make  a  choice  that  changes  the  meaning  of  the 
jj^riginal  sentence.         *      *  w^    ^  , 


ANSWER 


27. 


An  antipollution  act  intended  to  protect  Delaware  Bay  was  passed  by. 
■  the  state  legislature,  which  the  governor  'signed.  '  00 

m 

(A)  was  passed  by  the  state  legislature,  which  the  governor  signed  W 

(B)  was  passed  by  the  state  legislature  and  then? the  governor  signed  it  03 

{U 

(C)  was  passed  by  the  state  legislature  and  signed  by.the  governor 

(D)  and  which  was  passed  by  the  state  legislature,  was  signed^ by  the 
pov&raor  '  ' 

(E)  and  passed  by  the  state  legislature,  was  signed  ^y; the  governor 

Directions:     Thete  questions  ask  ypu  about  words  used  to  connect  ivdeas  in^ 
sentences.     You  are  to  choose  the  begt  word  or  phrase  to  use  ta  Join/the'' 
ideas  in  the  sentence  marked  I  to  the  ideas  in  the  sentence  iuarked  M; 
that  is,  you  are  to  choose  the  word  or  phrase,  that  will  make  the  mosj:  sense 
in  a  n^w  sentence  that  combines  sentences  I  and  II.     Some  sentences  Will 
have  tc  be  changed  only  a  little;  others  will  h^ve  to  be  changed  a 

great  deal.  \ 


37. 


Sarah  heard  about  the  tire  sale.  .  '  E 


II.  Sarah  left  to  buy.  new  tires. 

(A)  As  if 

(B)  Nevertheless  > 

(C)  However 

(D)  Despite^ 

(E)  As  soon  as  * 


The  new  sentence  is  "A.-;  soon  as  Snrah  heard  about  the  tire  sale,  she  left  to 
buy  iiev  tires."    or  "Sarah  leftn.0  buy  new  t.ltes  a.-;  soon  as  she  heard  about 
the  tiie  .sale."    Ansjwer  (L)  should  therefore  be  marked.  | 


121 

® 

m 


Figure  2.5(b) 


Although.  tSe -objective  ties ts  were  Jiisj:r*buteW- for' faculty  and  author 
review  in  the  spring  of  1975,   there  were  few Wcammfendat ions  for"  revisions . 
As  with  thi' preparation  o^f  math 'tests ,  sulfifiit ting  specif  ic  tests  for  participant 
review  had  avoided  unproductive ' requests  for  comment  an^  encouraged  constructive 
criticvism^Jt  also  facilitated  faculty  cooperation  and  contributed  to  the 
jests'   rel^abh4ty  and  validity.     Table  2;6  demons trat^^Pthis  point:     the  '■ 
objective  ,  tests  of  writing  skifls  had  respectable  coefficients  of  internal-  . 
consistency.     The  estimates  in  this  table  cover  th^Memonstrat ion  period  for  " 
t'he  TICCIT  program  in  English  compdsitioA#  For  ^idence  of  Validity  we  again 


cite  the  instructors'-  decision  to  cq^^M $eqj&g  procedures  established  f/r 
\our  evaluation  as  part  of  .their.  P^^j^^^fe  JICCIT -program  beyond  -the 
demonstration. 


Student  Attitudes,    ^Wlfifra^  student  survey  actually 

/  rv;  '    ■  ' 

included  questions  about  both^t^ftlts'   attitudes  toward  a  course  end  their 
•activities  related  to  jzhe  coutseu...  It  had  started*  as  just  a  survey  of  student 
attiXudes.     Indeed  the  fir^t \'^att  circulated  for  review  and  piloted  in  la?te 
1973  drew  ^heavily  from  instruments  concerned  with  student  ratings'  of  their 
instruction  (e.g.  ,  Centra, '19  72 )  .     It,  met  with  sound  criticism  from  students, 
faculty    'and  administrators y  There  was  too  much  emphasis  on  the  instructor..  In 
their  cJmments^  about/ the  questions,  however,  students  within  a  class  tended  to 
make  the  same  point.     This  suggested  the  power  of  one  person's  comment  "at  the 
time  of  survey  administration.     Still  the  cfiticisijD  was  well  taken:     there  was 
far  too  much  about  the  instructor  and  there  would  be  no  counterpart  for  most  of 
these  questions  in  a  survey  ^jn,t  ,-fibr,  classes,  under  the  TICCIT  program.  And 
perhaps  we  had  gotten  too  fat*  ahead  (of  the  TICCIT  program  for  it  was  still  under 
development  at  thlsr  time.,  '"•'<+ 
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Pretest  and  Postteat  Reliability:    Internal  Consistency  Esti™^ 

Alexandria  and  Phoenix 
English  Courses 
Calendar  Year  1976 


Academic  Term 
Spring  Semester 

Course 

Test 
Form 

Number  of 
Students 

Total 
Number 
of  Items 

r_ 

English  019 
English  029 

•  701  * 
701 

165 
118 

* 

40 
40 

.84 

.  0  1 

Summer  Semester 

English  029 

701 

74 

40 

.92 

Fall  Semester 

English  019 
English  029 

701 
701 

327 
152 

40 
40 

.86 
,.86 

Winter  Quarter 

English  111 

701 

184 

* 

40 

.85 

Spring  Quarter 

English  111 

701 

312 

40- 

.91 

Fall  Quarter 

English  111 

701 

259 

40 

.86 

Average : 

701 

Pretest 

40 

.87 

» 

Spring  Semester 

English  019 
English  029 

702 
702 

89 
58 

40 
40 

t 

.78 
.81 

Summer  Semester 

English  029' 

702 

41 

40 

.89 

FalWSemester 


% 


English  019 
English  029 


Winter  Quarter  ^English  111 

Spring  Quarter        English  111 

Fall  Quarter^       English  111 

\  ' 

\  Average : 


702 
702 

702 

702 

702 


231 

87. 

96 
248 
206 


702  Posttest 


40 
40 

40 

40 

40 

40 


.79 
.86 

'  .84 

.92 

.84 

.84 
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We  turned  to  studies  about  student  attitudes  toward  computer-assisted 
instruction  to  find  questions  without  as  much  emphasis  on  the  instructor.  What 
we  found  was  the  other  extreme:     either  comparison  between  conditions  was  not 
part  of  the  study's  purpose  (e.g.,  Summeriin,   1971)  or  the  statements  focused  on 
CAI  alone  (e.g.,  Mathis  et_ _al.  ,   1970).     An  instrument  had  to  be  prepared  that 
would  represent  a  compromise  between  ratings  of  instructors  satid  general  reactions 
to  CAI.     By  the  summer  of  1974  installation  of  the  TICCIT  system  had  begun  at 
the  colleges  and  two  parallel  student  surveys  were  sent  to  participants  in  the 
project  for  review.     One  form  was  intended  for  students  in  the  TICCIT  program 
and  another  for  students  in  regular  classes.     Each  form  covered  attitudes  toward 
the  course  as  well  as  activities  related  to  it. 

The  reception  and  reviews  for  these  two  forms  were  f avorab le .     It  was 
necessary  to  make  several  statements  more  specific  in  reference  to  capabilities 
of  the  TICCIT  system.     Another  refinement  replaced  the  constructed  responses 
about  interpersonal  contacts  with  a  frequency  scale  for  response  since  students 
balked  at  these  questions  in  survey  trials.     Essentially  these  were  the  forms 
used  in  the  second  half  of*  the  1974-75  academic  year.     The  only  additional 

changes  before  the  demonstration  phase  of  the  TICCIT  project  involved  improve- 

4 

ments,  primarily  to  the  survey  intended  for  lecture  classes,  so  that  statements 
would  be  identical  or  closely  parallel  across  conditions. 

The  questions  on  the  survey  dealt  with  common  claims  about  CAI,  specific 
features  of  the  TICCIT  program,  and  general  satisfaction  with  an  instructional 
condition.     Nearly  twenty  s  tatements  *were  the  same  \acr<^^onditions  or  just 
rephrased  to  fit  the  condition.     These  constituted  the  basis  for  our  comparisons 
of  program  results   in  terms  of  student  attitudes.     There  were  items  about  six 
specific;  feature?  of  TTCXTT's   instructional   design,  and  about   the  classroom 


ERLC 


-7  ' 

practices  that  best  approximated  five  of  these  six.     For  example,  practice 
problems  ("PRACTICE")  on  the  TICCIT  system  were  likened  to  homework  assignments 
in- regular  classes,  and  explanations  available  on  ^he  system  ("HELP")'  to  those 
given  by  an  instructor.     Other  items  simply  repeated>emmon  assumptions  about 
CAI  advantages,   about  instruction  that  meets  individual  student  n*eds  or  increases 
the  amount  of  attention  given  individual  students  and  so   forth,  and  disadvantages, 
such  as   the  impersonal  nature  of   this  method  of  in.truction.     Another  group  of 
statements  pertained   to  reactions  on  a  summary  level*.     These  posed  questions  on 
whether  students  would  recommend   the  course  to  their  friends  or  if   they  felt 
challenged   to  do  their  best-work  in  the  course.     Such  statements  called  for 
broad  judtments  of  course  quality  and  reflected  overall  satisfaction  with,  a 
course.     The  actual  survey   forms  for  which  we   report  data  appear  as  Appendix  F 
to   this  report. 

\      It  is  difficult  if  g»t  impossible  to  anticipate  all  student  reactions  with 
a  finite  set  of   questions.     Just  as  the  preparation  of   the  student  survey  had 
benefited  from  casual  conversations  with  students  and  their  written  comments  on 
preliminary  versions  of   the   instruments,   the  interpretation  of   survey  results 
might  also  benefit  from  informal  contacts  with  students  and  further  comment. 
Several  statements  on  the  survey  forms,   particularly  those  items  specific  to  an 
instructional  coalition,   had  been  suggested  by  students'   comments— their  wish 
'  for       textbook  to  use  at  home   in  TICCIT  classes  or  the  role  of  questions  raised 
by  other  students   in   lecture  classes.     We  did  not  want  our  reliance  on  an  ^ 
attitude  survey   to  preclude  comment  on  points  omitted   from  the   form  but  important 
to   stutrfents.      So  we  encouraged  students   to  elaborate  on  their  responses  and 
comment   freely  and   specifically  on  aspects  of   their  course.     Perhaps  because  it 
«L  a  m^ber  ul    the  evaluation  learn  who  administered   the  .student    survey,  students 
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provided  extensive  and  ,»eaniog£ul  comment.     ^  ^  be  presented  in  chapter  6, 
Student  Attitudes,  alDng  ^  the  data  from  the  ^  atem  on  the  survey. 

Table  2.7  gives  intetnal  consistency  estimateS>  based  on  an  aipha  coefficient 
(e.g.,  Cronbach,    1970),  ^  reLiability  ^  ^  courses.     For  each  form  of 

the  survey  the  several  of  reliabiUty  correspond  to  different  scores. 

The  total  score  is  the  ^  q£  a  student's  responses  w  all  attitudinal  stems. 
The  responses  themselv^  ^  t<>  be  choices  Dn  a  five_point  scale  of  agreement 
(strongly  agree  to  str0ftgly  disagree)  or  fr.^^  (aLmoSt  always  to  almost 
never).     The  surveys  lncUuM  ;)Lems  phrased   U}    {  positive  direction  for  which 
stronger  agreement  repr^^  a  more  positiVe  aM  ltude.  /  fnere  were  also 
stems  worded  in  a  negat^  Such  occa8loiial  shifts  in  stem  direction 

were  intended  to  disc ^  formation  of   response  sets  or  the  simple  ^peti- 
tion of  a  particular  cilo.^  ^  statements,  and  t0  foster  consideration  of 
each  item  as  a  separate  ^^t.     Ot  course  We  reversed   responses  to  negative 
item  stems  in  calculate,.                 and  excluded  responses  to  stems,  of  doubtful 
direction  from  the  scores>     T„ls   latter  type  0£   ^  was  meant  only  as  a  source 
of  information.     The  ave^  re,iability  for  sCore,    .90,   is  indeed  respect- 
able.    Hut  especially  Cot    ^  ,     tr ument  desi^   tQ  masuCe  attitudes  we  should 

look  at  reliability  l»  te,-        .  scales. 

litis  ei 

The  common  »™rv  ,„  Tabic  I.,  r,tl.I;;   ^  &  sCudunt's  score  on  those 

Items  common  across   i  IIa  L  f  ^  ,   conditions.  („ice  a>,aIn,   formed  ^ 

basis  tor  comparisons  (>,   ^  ^  results.     WiUu)U(;  tht.m  there  would  be  no 
standard   tor  gauging  wl,^,^       f,fVen  attitude  l()Ward  the  TICC1T  program  was  a  ^ 
positive  or  negative   re^    |fl  r,.|atl»«  to  ,mU;„   pra,Ll,-es.     The  average  re  11-  ' 
ability   for  this  score    ,„    ^  „,„„.    .8H.   .,„    „    8|i||uId  be   since    the  common 
"'•'»•"   i'«-lu,I..H  aluu.s.  (i  ,„,„„.    „t.mij   |h((    dHtinuilM.  reliability   ,,„■  total 
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Academic 
Term 

Spring  1975 


Fall  1975 


Winter  and 
Spring  1976 


Survey 
Forms 


Reliability  of  Student  Attitude  Survey: 
Internal  Consistency  Estimates 
Alexandria  and  Phoenix""" 
Mathematics  Courses 
1975-1976 

Number  of 
Respondents 


033-TICCIT 


032-Lecture 


28 


190 


035-TICC1T  (Phoenix)  144 
063-TICCIT  (Alexandria) 


'    040-Lecture  (Phoenix)  304 
062-Lecture  (Alexandria) 


069-  TICCIT  (Phoenix)  205 

070-  TICCIT  (Alexandria) 


040-Lecture  (Phoenix)  374 
062-Lecture  (Alexandria) 


Internal 


Number 

LOuSlotCUv/ 

Score 

of  Items 

r 

Total 

24 

Ql  A 

•  7  J  ™ 

Common 

1  A 
1^ 

HI  H 

Total 

1  ft 
15 

Common 

14 

.86 

Total 

24 

Common 

1  Q 

lo 

Satisfaction 

0 

Aft 

Attention 

4 

•  /  j 

System  Features. 

3 

.72 

* 

Total 

r  21 

.00 

Common 

10 

ft7 
.0/ 

Satisfaction 

<c 
0 

ftn 

Attention 

4 

7ft 

System  Features 

3^ 

'.64 

Total 

25 

•  yi 

Common  <• 

1  0' 

iy 

on 

Satisfaction 

0 

Attention 

A 

72 

System  features 

A 

.71 

Total 

21 

.88 

Common 

19  \ 

.87 

Satisfaction 

6 

.81 

Attention 

4 

.75 

System  Features 

4 

,67 

ERIC 


Table  2.7 
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score.     Factor  analyses  of  the  fall  and  spring  data  from  1975-76  suggested  that 
the  common  score  actually  consists  of  three  separate  scales  (the  analyses  and 
scale  descriptions  appear  in  Chapter  6:     Student  Attitudes).     These  scales  seem 
to  reflect  students'  overall  satisfaction  with  a  course,   their  opinions  about 
the  attention  given  to   them  in  the  course,  and  students'   reactions  to  specific 
features  of  the  TICCIT  system  alongside  other  students'   reactions  to  the  counter 
part  of   these  features  in   the  classroom.     Despite   the  small  number  of  items  that 
constitute  each  scale,   the  average  reliability  estimate  is  r  the  satis- 

faction scale,    .75  tor  the  attention  and   .68  tor  system  features.     These  alpha 
coefficients  for  reliability,  especially  with  regard  to  total  and  common  scores,, 
compare  favorably  with  estimates  reported  in  other  studies  of  attitudes  toward 
CAI    (e.g.,   Brown,    I960;   Williams  &  Milner,    l"76)  or  student   ratings  (e.g., 
Cost  in,   Creenough  k  Menges,  1971). 

Since  our  comparisons  of  program  results  will  be  on  the   level  of  a  course, 
it   is  also  appropriate   to   report   score  and  &i:alu  reliabilities  for  each  course. 
Tables   2.8  and  2.9  give   this    information   tor  math  courses  at  Phoenix  and 
Alexandria.     These  estimates  appear   to  be  consistent  with   those   for  all  courses 
together.     There    is  also  new  evidence  ot    reliability  given   tor  classes  at 
Alexandria.      that    college  had  srri  Ion;;  oi    one  math  course  under  programmed 
instruction.     The  survey  tor   this   instructional    condition  was  even  closer   to  the 
form  tor   the  TICCIT  program   hi   that    hot  h  were  geared   toward    independent  study 
programs   and  so  shared  a  nearly    identical    set    oi    item  st.-ms.      Still    the  common 
score  and  subsrales    fin  hide  only    items  shared   by  all    three   tonus    tor  TICCIT, 
programmed,   and    lectme  (lasses.      Tin-  survey   t  o  i  in  toi    programmed    instruction  has 
lower  estimates  ot    internal    ronsisteney   than    those    tor  all    courses    taken  together 
Hut,    the    slight    ditteteihe    is   piobahly    mlnm    rmu  i.r    I  In.  tn.it  ion    |usl     like  that 
for  other  courses  taken  one  at    a  time. 

/ 
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Reliability  of  Student  Attitude  Survey: 
Internal  Consistency  Estimates 
Phoenix 
Math  007,  106,  108,  and  117 
'Academic  Year.  1975-76 


Fall 


Score 
Iotal 

COMl 

Satisfaction 

Attention 1 

System 
Features 


TICCIT  Classes  J  Lecture  Classes 

Number  TICCIT  ta4 

of  Items  Hath  007  Kath  106  Math  108  Total  oMteto  Math  007  Math  106  Math  108 

(Form  035)  (N  *  27)  (H  *  20)  jl^H  (RjlSU.*  -S&M  &ZM  SLUM  UlSL  MM. 


Lecture 
Total 
(H  ■  260)* 


24 

.93  ' 

.91 

.94 

.92 

21  ' 

.87 

.85 

.87 

.83 

18 

.92 

.87 

"m 

.91    \  r 

18 

.86  " 

.84 

.86 

.87 

6 

.91 

.39 

.92 

.89 

6 

.77 

.77 

.83 

■  .80 

4 

.79 

.43 

.80 

.73 

4  'X 
i "  • 

.80 

.66 

.72 

.77 

.54 


.53 


.66  .66 


.62 


Spring 


■'  Total 
'  Common 
Satisfaction 

i 

Attention 

System 
Features 


(Form 00)  j^ffi  fi^UJ.        fcJil*  HiM  Hitt  ti&  111  UjLffl 


25 
19 
6 
4 


.94 

.92 

.90 

.92 

21 

.89 

.88 

.82 

(.89 

.88 

.94 

.91 

.88 

.91 

19  ' 

.89 

.87 

.81 

.88 

.87 

.90 

.88 

.83 

.86 

6 

.84 

.81 

.81 

.81 

.82 

.50 

.78 

.64 

.63 

4 

.73 

.75 

.70 

.72 

.73 

.82 

i 

.71 

.59 

.70 

4 

.67 

.71 

.54 

.65 

.67 

*Tctals  for  lecture  and  TICCIT  classes  include  students  from  Math  117. 
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Reliability  of  Student  Attitude  Survey: 
Internal  Consistency  Estimates 
Alexandria 
Math  31  and  32 
Academic  Year,  1975-76 


Score 

Total 
Common 
Fall  Satisfaction 
Attention 

System 
Features 


Total 
Common 

Winter  Satlsfaction 

Attention 

System 
Features 


Tdtal 
Common 
Spring  Satisfaction 
Attention , 

System 
Features 


TICCIT  Classes' 

Number 

of  Items      Math  31 
(Form  063)   ,(N  *  46)   Math  32 


24, 
18 

6 

4 


.90 
.90 
.88 
.73 

.82 


Total 
(N  »  80]* 

.90 
.88 
.87 
.61 

.76 


Programmed  Classes 
Number 

of  Items  Math  31 

(Form  064)  (N  ■  21) 


Lecture  Classes 

Number 

of  Items      Math  31 
(Form  062)    (N  ■  44)   Math  32 


24 

.92 

21 

.90 

18 

.89 

,  18 

.89 

6 

.84 

6 

.83 

4 

.75 

4 

'  .81 

.80 


.71 


1^0701  iN^64i  Jz^nL  Ji^ogy  iForn.OWl  ^N^      (Form  062)    ft  -  23)    (N  ■  2Q> 


25 
19 
6 
4 
4 


.91 
.89 
.84 
.73 
.76 


<Form  m)   i?L^i   (N  ■  18)    (N  *  42)       (Form  064)  (N 


38)      (Form  062)    (N  ■  12)  ' (N 


25 
19 
6 
4 
4 


.93 
.92 
.83 
.56 
.73 


.85 
.82 
.78 
.71 
.43 


This  number  includes  students  who  took  Math  18i  on  the  TICCIT  system. 


Total 


.91 

24 

.90 

21 

.87 

.94 

.91 

.90 

19 

.87 

19 

.85 

.94 

.90 

.87 

6 

.78 

6 

.71 

.83 

.78 

.68 

4 

.55 

4 

.78 

.86 

.82 

.72 

4 

.67 

4 

.77 

.74 

.76 

(N  ■  43)  i 

In 


30)     (N  -  42) 


.90 

24 

.91 

21 

.84 

.85 

.84 

.89 

19 

.89 

19 

.83 

.85 

.85 

.84. 

6 

.78 

6 

.75 

.70 

.72 

.63 

4 

.77 

4 

.83 

.68 

.72 

.66 

4 

.52 

4 

.49 

.57 

.56 

121 
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The  preceding  estimates  all  dealt  with  math  courses  while  Table  2.10  present 
comparable  data  for  English  courses.     Essentially  the  surveys  were  the  same  for 
both  subjects.    That  intended  for  composition  courses  did  include  another  two 
item  stems:     these  required  student  judgments  of  the  extent  of  their  learning 
about  grammar  and  composition.    Whether  the  TICC1T  program  would  promote  just 
the  learning  of  grammar  and  other  basic  skills  without  a  concomitant  increase  in 
demonstrated  writing  ability  was  a  central  question  to  the  English  departments 
involved  in  the  demonstration.     And  students'  own  judgments  were  important  to 
the  faculty  as  cbmf irmations  of  our  objective  and  essay  tests.     The  survey  for 
English  courses  naturally  referred  to  writing  skills  and  other  components  of 
composition  rather  than  mathematical  rules  or  problems.     The  changes  were 
minimal  and  thus  the  scales  and  reliabilities  for  the  attitude  survey  in  English 
courses  run  parallel  to  those  in  math.     The  common  score  again  seems  to  include 
scales  about  course  satisfaction,  attention,  and  specific  system  features.  And 
the  reliabilities  appear  consistent  with  those  for  math. 

student  Activities.     The  last  section  of  each  student  survey  concerned 
course-related  activities  (see  Appendix  F) .     To  label  all  the  questions  in  this 
last  section  as  pertinent  only  to  activities  would  be  too  restrictive.  What 
several  of  the  questions  tapped  might  be  interpreted  as  behavior  indicative 
of  affective  reactions:    plans  for  further  academic  work  in  the  same  subject 
(i.e.,  further  courses  in  a  math  sequence  f  additional  courses  in  composition 
or  literature)  and  the  appeal  of  a  job  related  to  the  course.     It  is  simply 
convenience  and  format  that  suggest  we  designate  such  questions  as  relevant  to 
activities.     Both  course  studies  and  employment  entail  future  activity,  and  a 
simple  dichotomous  (yes/no)  response  accompanied  these  questions  in  contrast  to 
the  Likert  scale  that  went  with  the  stems  of  attitudinal  statements. 


Reliability  of  Student  Attitude  Survey! 
Internal  Consistency  Estimates 
•    Alexandria  and  Phoenix 
English  Courses 
Calendar  Year,  1976 


Score 
Total 

Phoenix  Cmm 

English  19  Satiafaction 
and  i 
English  29  Attwtlon 

Systei 
Features 


Total 
Common 

Alexandria  Satlsfac«<'« 
English  111  Attention 

^  System 
Features 


TICCIT  Classes 

Nuiber  Spring  Fall  Across 
of  Item)   Semester  Semester  Terns 

flag  M  (N  ■  40)  (N  ■  lQQ)  (H  .  mi 


Lecture  Classes 

,* 

Number  Spring  hlii  Across  J 
of  Itess  Semester  Semester     Terms  j 

SLlM  W'lft)  (H'  233) 


27 

.84 

.87 

.87 

23 

.92 

.89 

\  t  -90 

21 

.83 

.8.7 

.86 

21 

.92 

.89 

.90 

8 

.  .82' 

.78 

.79  • 

8 

.85 

.79 

.81 

3 

.73 

.58 

.63 

3 

.70 

,  .77 

'.76 

k 

.62 

.73 

.70 

4 

.66  . 

.68 

,68 

Number    Winter    Spring      Pall  Across 
of  Items  Quarter  Quarter  Quarter  Terms 
JfiESM  S±M  SLLM  »  '  101)  (N  ■  211) 


Number  Winter  Spring  Fall  Across 
of  Items  Quarter  Quarter  Quarter  Terms 
(Form  0651  »  ■  43).  IN  ■  140)  (N  ■  63[  (N  ■  248) 


27 

.91 

.89 

■  .89 

.90 

23 

•91 

.87  . 

.90  , 

.89 

.  .88 

21 

.90 

.87 

.88' 

.89 

•  *  *n 

.90 

.86 

.90 

8 

.8(| 

.73 

.78 

.78 

'8 

.80 

.78 

,78 

"  .78 

3 

.n 

•75 

■  .74 

.74 

3 

.69 

.67 

.74 

.70 

4 

.83  " 

.80 

.72 

.77 

4 

.80 

.81 

.77 

'.80 

Table  2,10 
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There  were  just  two  other  questions  in  the  section  of  the  surveys 

activities.     One  multi-part;  question  dealt  primarily  with  the  frequency ji?f  a 

student's  interpersonal  contacts  related  to  the  course.     Its  components  covered 

discussions  with  classmates  about  course  work,  meetings  with  the  instructor 

* 

outside  of  class  time  about  academic  matters,  attempts  to  obtain  tutorial  .? 

t 

assistance  and  even, the  use  of  campus  resources  like  the  library.  Another 

•  *'a 

multi-part  question  required  that  a  student  estimate  the  number  of  hours  per 
week  that  he  had  devoted  to  various  course-related  tasks,  such  as  class  time  \ 
and  homework  assignments.     Both  of  these  last  two  questions  plainly  concern  ^ 

activities.  v  *\. ,  \  JlV 

These  few  questions  from  the  last  section  of  the  student  survey  cj^ticuj&  .. 
the  extent  of  our  study  of  student  activities  across  instructional  cWitionsV 
Obviously  our  intent  was  much  more  modest  than  that  involved  in  the  study  of  ^ 
either  achievement  or  attitudes.     Yet  students'  plans  for  further  course.  wor# 
and  the  attractiveness  of  a  job  related  to  the  subject  matter  represented  one  * 
dimension  of  the  goals  of  the  TICCIT  program  and  so  deserved  to  be  choked  at  V 
least  at  this  minimal  level  of  effort  arid  cost  to  the  evaluation.  Similarly,^.^ 
critics  often  claim  that  CAI  detracts  from  rather  than  enhances  interperson^/ 
contacts  and  this  too  warranted  some  independent  check.     The  issue  rff  student 
study  time  is  not  as  minor.     For  time  spent  in  learning  bears  on  llff^^e/, 
theory  and  policy.     If  the  TICCIT  program  did  driver  instruction  tailok^d  to 
the  individual  student,  it  might  precipitate  a  break  with  the  academic  term  and 
enable  some  or  most  students  to  finish  a  course  in  far  less  time.     This  efficien 
might  in  turn  force  a  college  to^consider  departures  from  the  time  constraints 
of  an  academic  term,  especially  if  there  were  a  strong,  relationship  between  time 
and  learning,  in  the  TICCIT  program.     Then  too,  there  was  the  matter  of  what 
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students  actually  did  on  a  TICCIT  system  to  take  advantage  of  its  learner 
control  and  what  strategies  for  learning  evolved.  L"1 

Of  the  resources  expended  in  documenting  student  activities  most  went 
toward  defining  what  happened  on  the  TICCIT  system,    A  special  rdport  gave  a 
history  of  each  student's  use  of  learner  control  for  every  lesson  and  unit  in 
his  course.     It  also  recorded  his  performance  on  practice  problems  and  the 
amount  of  time  spent  in  every  lesson.     Regretfully  these  summaries  proved  to  be 
unreliable:     sometimes  hu^reds  of  access  requests  appeared  where  fewer  than  one 
hundred  were  possible,     Th^Tault  might  rest  with  the  special  computer  program 
prepared  independently  of  the  sys tem' s  developers ,  with  lapses  in  the  regularity 
of  updates  to  the  data  base  since  these  consumed  much  computer  time,  or  perhaps 
early  difficulties  with  the  student  records  maintained  on  the  system.     Still,  we 
will  take  advantage  of  these  student  histories  to  explore  interrelationships 
among  TICCIT  features  and  to  give  a  picture  of  the  massive  use  of  the  TICCIT 
program  that  occurred  on  a  course  level. 

Another  report  generated  on  the  TICCIT  system  listed  students'  progress  on  , 
a  lesson  by  lesson  basis.     It  had  less  detail  than  the  report  specially  produced 
for  the  evaluation,  but  it  was  reliable.     The  weekly  progress  report  had  been 
designed  and  developed  by  the  system's  developers.     Since  it  was  intended  as  a 
management  report  for  teacher  use  in  counseling  it  was  subject  to  instructors' 

•Vfc, 

scrutiny  for  accuracy.     In  the  first  term  of  the  demonstration  period  there  were 
inaccuracies-  and  instructor  criticisms.     These  subsided  as  corrections  were  made 
in  the  fall  of  1975.     Thereafter  the  last  progress  reports  for  an  academic  term 
became  our  source  for  data  on  time  students  spent  on  the  system  and  on  the 
number  of  lessons  passed  by  each  student.     These  data  on  time  and  mastery  will 
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be  joined  with  our  own  pretest  and  posttest  achievement  scores  in  an  attempt  to 

) 

look  at  time  and  learning  on  the  TICCIT  system. 

Finally,  we  undertook  direct  observations  of  students  studying  on  the 
TICCIT  system.     The  observations  concentrated  on  the  extent,  pace,  and  sequence 
involved  in  students'  use  of  learner  control.     Particular  emphasis  was  on  the 
use  of  rules  ("RULE"),  examples  of  rule  applications  ("EXAMPLE"),  practice 
problems  ("PRACTICE"),  and  explanations  available  on  the  system  ("HELP").  This 
also  afforded  an  opportunity  to  document  students'  calls  for  optional  system 
features,  like  management  advice  on  how  they  might  proceed  wit&  their  study  on 
TICCIT  ("ADVICE"),  as  well  as  their  requests  for  assistance  from  other  students, 
proctors  or  instructors. 

Over  the  spring  and  .summer  terms  of  1975  the  procedures  for  observing 
students  went  through  a  period  of  trial  and  refinement.     By  the  fall  the  instru- 
ment was  ready  as  well  as  explicit  instructions  for  coding.     Appendix  G  contains 
the  coding  form  for  observations  of  student  interactions  with  the  TICCIT  system 
and  the  instructions.     Since  two  different  members  of  the  evaluation  team 
observed  students  working  at  terminals,  both  conducted  a  common  set  of  obser- 
vations as  a  check  on  inter-observer  reliability.     For  the  same  twenty  student 
sessions  at  terminals  the  observers  assigned  the  same, codes  over  90%  of  the 
time.     And  nearly  half  of  the  disagreements  were  traced  to  an  ambiguity  in  the 
instructions  and  resolved:    one  observer  had  counted  all  student  use  of  system 
explanations  while  the  other  counted  only  that  use  associated  with  practice 
problems.     Another  quarter  of  the  discrepancies  in  coding  were  due  to  one 
observer's  "note  pad"  being  the  other's  "work  sheets"  in  checking  materials 
present  at  a  terminal.     Details  of  these  results  appear  in  Appendix  G  along  with 


the  coding  form  and  instructions.  Results  of  the  observations  come  in  Chapter  7: 
Student  Activities. 

Student  Characteristics.     One  last  instrument  pertinent  to  student  perfor- 
mance served  as  a  record  of  what  variables  students  brought  with  them  into  the 
instructional  conditions.     Even  if  random  assignment,  blind  section  selection, 
or  haphazard  registration  worked  in  our  favor  and  resulted  in  roughly  equivalent 
comparison  groups,  there  was  no  guarantee  that  groups  would  still  be  equivalent 
at  the  end  of  the  term.    We  thought  it  advisable,  therefore,  to  introduce  a  check 
on  enrollment  procedures  and  gather  data  about  student  characteristics.    At  the 
very  beginning  of  each  terms  we  asked  that  all  students  in  target  courses 
complete  a  brief  questionnaire  (see  Appendix. H). 

The  questionnaire  was  prepared  to  find  out  why  students  chose  a  particular 
section  of  a  course  as  well  as  get  a  profile  of  those  students  enrolled  under  an 
instructional  condition.     In  documenting  section  selection  we  tried  to  discover 
whether  anyone  had  recommended  the  particular  teaching  method  to  students  and 
what  reasons  played  important  roles  in  students'  decisions  to  enroll  in  a  section 
(e.g.,  other  sections  already  full,  preference  for  a  particular  instructor). 
Also  we  wished  to  receive  data  on  students'  demographic  characteristics.  These 
data  had  two  functions.     The  first  was  as  a  check  for  gross  differences  between 
comparison  groups  at  the  start  of  a  course.     But  even  grpss  differences  uncor- 
rected with  our  dependent  measures  (completion,  achievement,  attitude)  would 
not  bias  the  results.     It  was  important  to  have  data  on  student  characteristics 
not  only  as  an  initial  profile  of  comparison  groups  but  also  as  a  means  for  making 
adjustments  in  our  analyses  and  exploring  possible  program  effects  dependent  on 
student  traits. 


The  set  of  demographic  variables  included  in  our  data  base . is  admittedly 
a  rather  limited  one.     It  covers  high  school  and  college  grade  point  averages, 
number  of  college  credits  earned,  type  of  high  school  preparation  (i.e.,  general, 
vocational,  academic,  equivalency  test),  student  status,  employment  commitment, 
age  and  sex.     By  design  it  does  not  contain  variables  on  either  socioeconomic 
or  ethnic  status.     The  student  population  at  community  colleges  seems  too  hetero- 
geneous for  us  to  pose  simple  question  about  socioeconomic  status,  and  the 
initial  purposes  of  the  evaluation  bear  no  relation  to  ethnic  status.     In  general, 
we  tried  to  design  an  evaluation  that  would  intrude  as  >^tte  as  possible  on  the 
conduct  of  normal  college  services  and  the  privacy  of  students  and  faculty. 

2.2.b     Faculty  Acceptance  and  Teacher  Role 

The  purposes  of  the  evaluation  were  largely  descriptive  when  it  came 
to  documenting  the  impact  of  the  TICCIT  program  on  faculty.     We  sought  to 
describe  what  instructors  did  in  TICCIT  classes  and  how  they  reacted  to  the 
project.     There  were  no  comparison  groups  of  instructors  responsible  solely  for 
the  TICCIT  program  or  traditional  practices.     Such  strict  control  over  faculty 
would. only  confound  those  effects  due  to  the  programs  with  those  due  to  different 
instructors.     Instead  instructors  had  a  combination  of  classes,  some  under  the 
TICCIT  program  and  other  sections  iri  a  lecture-discussion  format.     And  hence, 
there  will  be  no  estimates  of  an  effect  on  faculty  in  the  same  sense  as  an 
effect  on  students.     The  effects  described  here  reflect  TICCIT's  impacts 
teachers  and  teaching  as  faculty  attitudes  toward  the  program  and  instructors 
estimates  of  their  activities. 

What  differences  there  were  in  faculty  exposure  'to  the  TICCIT  program  arose 
naturally  from  the  project.     Only  instructors  from  the  math  and  English  depart- 
tnents  actually  participated  in  the  TICCIT  program  since  the  target  courses  were 
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in  algebra  and  composition.    Other  faculty  members,  with  the  exception  of  the 
project  directors  at  each  college,  had  at  best  indirect  involvement.  While 
there  would  be  thousands  of  students  involved  in  the  TICCIT  demonstration, 
there  would  be  at  most  sixty  instructors.    With  so  small  a  group  and  a  number 
of  diverse  factors  other  than  the  evaluation  to  be  considered  in  teaching 
assignments,  there  could  be  no  comparison  groups  among  faculty  that  would 
correspond  to  the  teaching  condition.     Nor  would  such  groups  be  appropriate 
given  student  performance  as  the  foremost  consideration  in  evaluating  program 
results.     But  there  would  still  be  academic  departments  with  different  degrees 
of  exposure  and  course  sections  under  different  instructional  conditions.  So 
it  was  possible  to  explore  impact  by  looking  at  attitudes  across  departments  and 
at  teaching  activities  by  type  of  section. 

Even  on  a  descriptive  level  our  evaluation  depends  on  some  form  of  com- 
parison.    For  faculty  acceptance  we  can  contrast  attitudes  toward  CAI  in  general 
with  latetf  reactions  to  the  TICCIT  progifem  in  particular.     This  implies  at  least 
two  surveys:     one  about  CAI  generally  and  another  about  the  TICCIT  program 
specifically.     On  each  survey  we  can  also  examine  opinions  from  the  perspectives 
of  different  respondents  such  as  administrators  and  instructors,  or  instructors 
responsible  for  students  in  the  TICCIT  program  and  others  without  such  direct 
experience.     This  latter  contrast  along  with  changes  in  faculty  opinions  over 
time  provides  some  evidence  of  TICCIT's  impact  on  acceptance.     For  activities 
the  contrasts  follow  from  teaching  assignments.     Faculty  estimated  for  each 
section  the  percet^pge  of  their  time  spent  on  various  teaching  duties.     We  can 
then  take  these  estimates  aggregated  by  type  of  section  and  compare  an  average 
distribution  for  a  TICCIT  class  against  that  for  a  regular  section. 
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These  reactions  to  the  program  and  estimates  of  time  allocations  represent 
Judgments  as  much  mm  impact.     Indeed,  there  were  no  program  outcomes  with 
facuLty  in  the  same  sense  as  outcomes  like  student  achievement  or  course  com- 
pletion rates.     The  formation  of  opinions  is  part  experience  or  knowledge  about 
a  program  and  part  professional  Judgment  that  draws  on  other  experience  and 
knowledge.     Decisions  about  devoting  time  to  conflicting  demands  reflect  student 
needs  unfulfilled  by  a  curricular  program  and  instructors'   judgments  on  how 
they  might  best  meet  such  demands.     Whether  we  should  attribute  attitudes  and 
activities  to  impact  of  the  TICCIT  program  or  judgments  of  faculty  perhaps 
depends  on  the  extent  of  consistency   in  results  across  colleges  and  within 
subgroups  (e.g.,  the  math^and  English  .departments) .     Consistency  would  suggest 
impact  free  of  differences  due  to  individuals'  judgments. 

Special  instruments  best  suited  our  purposes  in\describing  attitudes  and 
activities.     But  our  decision  to  engage  In  further  instrument  preparation  was  as 
much  a  matter  of  necessity  as  of  choice.     A  standardized  survey  would  afford  the 
advantages  of  norms  and  enable  us  to  avoid  preparing  instruments  beyond  those 
required  for  students  performance.     A  widely  available  questionnaire  on  faculty 
activities  would  probably  provide  reference  points  on  practices  followed  e^ewhere. 
There  is,  however,  insufficient  need  for  surveys  or  questionnaires  about  CA1  and 
faculty  acceptance  or  activities,  and  even  less  for  instruments  concerned  with  a 
specific  program.     Some  special-purpose  surveys  did  exist   (e.g/Nlussell,  1969) 
but  none  that  matched  the  context  of  the  TICCIT  demonstration  well.  That 
context  involved   teaching  full  courses  by  computers  at  community  colleges,  and 
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it  required  that  a  faculty  attitude  survey  and  activity  questionnaire  be  developed 
if   the  evaluation  was  to  describe  faculty  acceptance  and  teacher  role. 


Faculty  Attitudes*     A  description  of  the  initial  development  and  report  of 
the  first  results  from  a  faculty  attitude  survey  appear  in  Alderman  and  Mahler 
(1977).     That  survey  dealt  with  attitudes  toward  certain  educational  practices 
as  well  as  computer-assisted  instruction.     However,   there  had  been  a  low 
response  rate  to  that  survey  and  a  few  of  the  dimensions  given  for  attitudes 
toward  CAI  seemed  to  be  artifacts  of  item  format.     So  another  general  survey  was 
conducted  in  the  spring  of  1974,  a  year  after  the  original.     The  revised  instru- 
ment (see  Appendix  I)  had  fewer  statements  and  concentrated  on  educational 
practices  pertinent  to  the  TICCIT  program,  such  as  mastery  learning  and 
self-paced  instruction,  and  general  assumptions  about  CAI,  for  instance  its 
potential  to  tailor  instruction  for  individual  students.     The  return  rate  for 
this  second  survey  was  over  twice  as  high  as  that  obtained  with  the  first:  76 
percent  of  all  faculty,  or  239  persons,  returned  completed  surveys.  Perhaps 
the  anonymous  procedures  for  response  and  the  shortened  form  (50  fewer  questions 
than  the  original  form)  contributed  to  this  improved  rate  of  return. 

A  third  survey,  also  in  Appendix  I,  was  sent  to  all  faculty  at  the  partici- 
pating colleges  during  the  first  academic  year  of  the  demonstration  phase  for  the 
TICCIT "project .     This  was  nearly  two  years  after  the  successful  general  survey, 
and  after  more  than  four  academic  terms  of  faculty  exposure  to  the  TICCIT  program 
for  math  courses  (and  just  as  composition  courses  on  the  system  got  under  way). 
Essentially  the  items  on  this  form  were  statements  from  the  general  version 
reworded  so  as  to  refer  to  the  TICCIT  program  specifically  along  with  several 
statements  taken  from  the  student  attit.  \o.  survey.     This  composition  permits  us 
to  contrast  general  attitudes  toward  CAI  with  subsequent  reactions  to  TICCIT,  and 
to  note  a  consensus  or  discrepancy  between  student  and  faculty  perceptions.  Over 
200  faculty  members  again  responded  to  our  attitude  survey. 
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Both  the  general  and  specific  survey  adhered  to  the  item  format  used  in 
collecting  data  on  student  attitudes.     Each  item  was  a  statement  accompanied  by 
a  five-point  response  scale  for  indicating  extent  of  agreement.     There  were  30 
such  items  on  the  general  survey  and  35  on  the  version  about  the  TICCIT  program 
in  particular.     The  statements  themselves  reflected  mostly  positive  attitudes 
toward  CAI  and  TICCIT  such  as  "TICCIT  helps  to  make  better  and  fuller  use  of 
instructors'  capabilities."    There  were  also  several  statements  phrased  in  a 
negative  manner,  "TICCIT  is  a  potential  threat  to  the  jobs  of  faculty  members." 
These  intermittent  reversals  were  intended  to  help  respondents  make  distinctions 
among  statements  (and  discourage  response  sets).     But  the  section  of  the  general 
survey  that  concerned  educational  practices  really  had  no  direction:     we  do  not 
prei-nd  to  know  positive  from  negative  practices.     Instead  this  section  merely 
documented  faculty  opinions  about  concepts  related  to  the  TICCIT  program.  The 
specific  survey  also  included  a  question  about  respondents'  experience  with 
TICCIT.     Through  this  question  we  could  identify  those  instructors  most  familiar 
with  the  program  and  report  their  reactions  apart  from  oVerall  opinions  at  the 

colleges.  t.  t 

As  with  the  student  survey,  estimates  of  reliability  for  the  faculty 

t' 

attitude  measures  should  refer  to  scales.     The  scales  for  which  Table  2.11 
gives  internal  consistency  (alpha)  coefficients  correspond  -to  the  dimensions 
established  through  factor  analyses.     In  the  absence  of  extensive  evidence  for 
validity  these  scales  merely%epresent  a  convenient  shorthand  for  describing 
faculty  opinions  and  drawing  contrasts  among- groups  of  respondents.     The  state- 
ments encompassed  by  a  scale  and  additional  detail  on  the  factor  analyses  may  be 
found  in  Chapter  8:     Faculty  Acceptance  and  Teacher  Role.     It  is  sufficient 
just  to  note  the  reliability  estimates  for  the  faculty  attitude  surveys:  the 
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Reliability  of  Faculty  Attitude  Survey 
Internal  Consistency  Estimates 
Alexandria  and  Phoenix 
1974  and  1976 


Survey  Form 

002-Initial 
Attitudes 
(1974) 


Number 
of  Respondents 

209 


003-Attitudes 

toward  TICCIT 
(1976) 


184 


Score 


Number 
of  Items 


Educational  Practices  17 

Student  Focus  6 

Subject  Matter  Emphasis  5 

Receptivity  2 

Incentives  3 

Computer-Assisted  Instruction  13 

General  2.0 

Familiarity          x  2 


Familiarity  2 

TICCIT  33 

Global  Judgments  18 

Management  7 

Limitations  4 

Faculty  Role  2 


Table  2.11 
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coefficient  is  high,  as  high  as  .93,  except  for  those  scales  with  .very  few 
items.     A  validity  study  for  these  scales  was  beyond  the  scope  of  our  evaluation. 
However,  the  statements  within  each  dimension  and  differences  found  among  faculty 
members  at  least  suggest  that  the  dimensions  represent  meaningful  constructs. 

Faculty  Activities.     A  questionnaire  about  faculty  activities  appears  in 
Appendix  J.     The  format  and  activity  breakdown  follow  from  those  employed  in  an 
evaluation  of  another  non-traditional  college  program  (i.e.,  Hartnett,  Clark, 
Feldmesser,  Gieber  &  Soss,   1974).     The  questionnaire  required  instructors  to 
report  on  their  use  of  classroom  hours  as  well  as  the  distribution  (in  percentages) 
of  their  time  spent  in  different  activities.     These  hours  and  percentages  were  to 
refer  to  a  regular  work  week  in  each  section  of  a  course.     Thus  instructors  might 
complete  from  zero  to  five  of  these  estimates  depending  on  the  number  of  sections 
of  target  courses  each  instructor  had  in  an  academic  term.     The  responsibilities 
or  activities  listed  in  the  questionnaire  ranged  from  planning  for  class  to 
counseling  students  and  taking  attendance,  and  included  conducting  class, 
preparing  lectures  and  other  duties  associated  with  teaching. 

There  was  a  second  part  of  the  activity  questionnaire  unrelated  to  specific 
sections.     It  concerned  general  activities,  such  as  advising  student  organizations 
or"revising  curricula  or  meeting  for  college  committees,  that  might  represent  a 
significant  redistribution  of  time  as  a  result  of  the  TICCIT  program  outside  the 
confines  of  a  section  assignment  for  teaching.     The  questionnaire  ended  with  a 
request  that  instructors  indicate  their  perceptions  of  the  relative  advantages 
offered  by  the  TICCIT  program  and  regular  practices  for  instruction,  and  a 
couple  of  questions  about  instructors'  orientation  to  the  TICCIT  system. 

The  questionnaire  on  faculty  activities  was  not  a  major  part  of  our 
evaluation.     It  was  meant  only  to  describe  what  role  instructors  assumed  with 
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the  TICCIT  program  as  reflected  in  their  own  estimates  of  time  devoted  to 
classes  on  the  computer  system.     As  noted  earlier  in  this  section  on  instrument 
preparation  for  faculty  attitudes  and  activities,  these  self-reports  from 
instructors  incorporate  elements  of   .  ^ -  "  and  judgment.    Another  college's 
faculty  might  adopt  procedures  di     erern;  f rom  those  at  Alexandria  or  Phoenix. 
Maybe  teacher  role  contributes  as  mu< h   i-    defining  the  context  of  the  TICCIT 
demonstration  as  it  does  to  documenting  possible  program  outcomes.     Indeed,  we 
will  refer  often  to  the  instructor's  part  in  the  TICCIT  program  in  explaining 
results  for  student  performance.     Such  reference  is  less  a  function  of  the 
instructor  activity  questionnaire  than  direct  observation  of  comparison  con- 
ditions and  regular  contact  with  instructors.     Data  from  instruments  may  help  us 
in  identifying  effects,  but  it  is  our  knowledge  of  the  full  program  conditions 
that  helps  in  understanding  what  the  effects  mean.  *  V 

2.3    Data  Collection 

Certainly  the  complications  that  might  be  expected  in  collecting  data  under 
field'  conditions  arose.     At  one  point  in  the  evaluation  forty  posttests  were 
mislaid  for  months  and  at  another  all  pretests  for  one  course  were  lost  in  the 
mail.     The  mislaid  posttests  were  found  filed,  incorrectly,   in  a  secure  location 
at  one  college.    As  a  replacement  for  the  lost  pretests  we  turned  to  records  of 
test  scores  kept  at  the  colleges.     Then  there  were  other  obstacles  to  a  truly 
planned  experiment:     a  iew  members  of  the  English  faculty  were  reluctant  to 
administer  objective  tests,  and  their  reservations  were  strong  enough  to  keep 
them  from  doing  so  during  the  demonstration;  some  part-time  faculty  were  unfamiliar 
with  the  project  and  our  data  requirements  and,  hence,   left  inadequate  class  time 
for  posttests  or  surveys.     In  short,  the  conditions  that  differentiate  a  field 
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study  from  a  planned  experiment  affected  our  evaluation.     But  these  had  a 
negligible  impact  and,  we  believe,  even  as  a  cumulative  obstacle  to  data 
collection  introduce  no  bias  in  our  results. 

These  inevitable  disruptions  seemed  to  occur  on  the  basis  of  a  course 
section  but  our  unit  of  observation  was  the  student.     Each  individual  student 
enrolled  in  a  course  on  the  TICCIT  system  went  through  a  curricular  program 
unique  to  that  student.     Because  students  could  exercise  control  over  various 
features  of  the  TICCIT  program  and  because  computer  systems  do  tailor  instruc- 
tion,  the  presentation  of  material  varied  by  student.     The  unit  of  observation 
for  the  TICCIT  program  was  clearly  the  student.     Although  other  studies  may  use 
the  classroom  as  the  appropriate  unit  for  lecture  sections  (e.g.,  Page,  1958), 
here  it  had  to  conform  with  the  unit  for  observing  outcomes  under  the  alternative 
program. 

.  This  choice  of  the  student  as  our  unit  of  observation  affected  other 
activities  in  the  evaluation.     It  meant  that  the  student  was  also  the  unit  for 
random  assignment  to  conditions  where  we  succeeded  in  arranging  such  procedures. 
Of  course  data  collection  and  also  analysis  followed  with  the  student  as  the 
appropriate  unit.     Thus  when  a  student  completed  his  work  on  linear  equations, 
for  example,  it  was  time  for  him  t^take  the  topical  test  on  that  material.  It 
happened  that  all  students  in  a  lecture  section  finished  at  the  same  time  but 
rates  differed  "for  students  in  a  TICCIT  class.     With  regard  to  analysis  it  was 
the  number  of  students,  not  sections  or  schools,  that  determined  sample  size. 

2. 3. a  Sampling 

In  a  sense  there  was  no  sampling  for  this  evaluation.     Data  were  to  be 
collected  on  the  performance^  all  students.     But,  except  for  completion  rates 
determined  by  official  grade  sheets,  this  was  impossible.     Some  students  register 
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for  tour s es  and  then  withdraw  before  attending  a  single  class.     Others  start  a 

:>.  .  .  *■ 

course  and  fail  to  satisfy  all  requirements  for  receiving  credit.     Thus  the 
number  of  pretests  given  in  a  class  will  be  lower  than  the  total  student  enroll- 
ment for  that  class,  and  the  number  of  posttests  lower  still  due  to  attrition 
and  incomplete  work.     But  of  those  students  who  received  credit  for  a  course, 
ovgr  90%  also  took  our  posttest  even  where  these  tests  were  not  adopted  by 
department  policy  as  final  exams. 

There  were  naturally  exceptions  to  our  otherwise  successful  data  collection, 
and  some  affected  the  sample  of  students  for  whom  complete  data  were  available. 
Attendance,  specifically  absence  of  students  at  the  time  of  pretest  adminis- 
tratiori,  caused  the  loss  of  a  few  cases  of  paired  pretest-posttest  observations 
from,  our  achievement  analyses.     By  far  the  greatest  influence  on  our  sample  was 
what  Campbell  and  Stanley  (1963)  call  "mortality."    Between  the  onset  of 
ins truction , and  the  end  of  an  academic  term  a  sizable  percentage  of  students 
fell  behind  in  their  course  work  and  therefore  received  no  credit  and  took  no 

j>osttest.     But  this  finding  and  the  study  of  differential  completion  rates 
across  conditions  were  part  of  the  evaluation  (see  Chapter  4:     Course  Completion 
Rates).     It  is  sufficient  here  to  stress  the  importance  of  this  aspect  of 
student  performance  for  its  possible  impact  on  other  outcomes. 

Experience  during  field  trials  did  force  us  to  reconsider  sampling  plans 
for  the  math  courses  at  one  college.     The  enrollment  trend  for  one  algebra 
sequence  was  on  a  steady  and  sharp  decline.     Further  there  was  a  poor  overlap 

•between  objectives  of  the  course  as  taught  with  the  TICCIT  program  and  in  the 
classroom. -  This  course  sequence  was  dropped  from  the  evaluation  of  the  demon- 
stration period  for  the  TICCIT  project.     In  another  course  sequence  the  initial 
enrollment  together  with  the  course  completion  rates  left  few  students  to 
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rep^sent  program  effects  on  their  performance.     So  we  decided  to  administer 
posttests  to  all  students  still  attending  classes  at  the  end  of  a  term,  regard- 
less of  their  probable  completion  status.     This  meant  that  even  those  students 
making  insufficient  progress  toward  course  completion  took  the  posttest.  This 
may  seem  like  an  unreasonable  population  to  study  but  there  would  be  course 
grades  to  distinguish  among  groups  (i.e.,  only  those  students  who  had  received 
credit  as  opposed  to  all  students  regardless  of  grade)  and  these  students  gave 
us  data  on  a  broader  population,  in  effect  an  estimate  of  overall  program  impact 
on  student  achievement  within  an  academic  term  and  independent  of  grades.  The 
unacceptable  alternatives  were  to  risk  inadequate  sample  size  or  not  to  evaluate 
the  TICCIT  math  program  at  one  site. 

Generally,  given  those  students  who  start  a  course  and  those  who  receive 
credit  as  the  populations  of  interest,  the  samples  in  our  evaluation  constitute 
nearly  the  entire  populations.     Pretesting  and  posttesting  became  a  part  of  the 
target  courses.     Instructors  expected  members  of  the  evaluation  team  to  come 
into  their  classes  in  order  to  distribute  and  collect  attitude  surveys  at  the 
end  of  a  course.     The  cooperation  of  instructors  at  both-  colleges  was  clearly 
indispensable  in  this  process,  and  their  support  of  our  data  collection  was 
exemplary.     Faculty  worked  with  us  toward  assigning  students  randomly  to  con- 
ditions and  collecting  data  on  student  achievement  and  attitudes. 

Observations  of  student  activity  and  classroom  conditions  necessitated  a 
much  different  form  of  sampling.     The  resource  demands  of  observing  students 
studying  on  the  TICCIT  system  made  us  choose  between  tracking  certain  students 
throughout  a  course  and  trying  to  cover  all  students  perhaps  just  once.  Again 
experience  from  trials' in  the  year  preceding  the  demonstration  guided  our 
decisions.     At  each  college  representatives  for  the  evaluator  randomly  selected 
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about  80  students  registered  for  the  same  course  on  the  system.     It  took  about 
two  weeks  to  complete  a  cycle*  of  observations  and  there  were  up  to  five  obser- 
vations on  each  student,  dependent  on  his  continued  and  regular  attendance. 
These  direct  observations  allowed  us  to  watch  for  changes  in  students'  use  of 
the  system  over  an  academic  term.     Later  access  to  online  data  summaries  "freed 
the  evaluation  staff  for  closer  supervision  of  expanded  data  collection  on 
student  achievement  and  other  duties.     For  they  also  observed  each  regular 
section  of  a  target  course  once  or  twice  in  an  academic  term  and  each  TICCIT 
class  several  times.     There  was  no  systematic  basis  for  these  class  observations 
since  they  were  intended  to  be  descriptive. 

2.3.b  Schedule 

Our  sampling  could  take  place  only  if  the  colleges  decided  to  offer 
courses  on  the  TICCIT  system.     In  turn  such  college  decisions  depended  on  the 
availability  of  a  fully  operational  TICCIT  program.     Obviously,  then,  the  schedul 
for  the  evaluation  was  not  ours  to  determine  but  instead  followed  from  the 
colleges'  activities  and  developers'  progress.     At  first  the  colleges  offered 
only  a  few  course  sections  on  the  TICCIT  system.     This  implementation  phase  of 
the  TICCIT  project  served  as  a  trial  period  for  instruments  and  data  collection 
procedures.  As  the  colleges  gained  experience  and  confidence  in  the  TICCIT 
program,  additional  sections  were  scheduled  on  the  system,  both  in  courses  ^ 
already  attempted  and  new  target  courses.     When  nearly  half  of  a  course's  total 
sections  were  on  the  TICCIT  system  the  evaluation  of  program  impact  began. 

The  schedule  for  our  data  collection  appears  in  Figure  2.6.     At  least  one 
academic  term  of  college  exposure  to  a  particular  course  on  the  TICCIT  system 
always  preceded  our  collection  of  data  on  program  impact.     This  period  of 
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Data  collected  in  implementation  period. 

Data  collected  for  evaluation  of  program  impact. 

Data  collected  in  course  sections  on  the  TICCIT  system  only. 

Continuation  of  college  use  of  the  TICCIT  program  beyond  data  collection. 
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adju8tment  separated  our  documentation  of  system  installation  and  initial 
program  trials  (see  Chapter  3:     Implementation  of  the  TICCIT  Program)  from  our 
evaluation  of  results  (see  Chapters  4  through  7  on  student  performance) •  For 
math  courses  the  demonstration  phase  of  the  project  began  with  the  fall  term  of 
the  1975-76  academic  year;  it  commenced  with  the  start  of  the  19  76  calendar  year 
for  English  courses •     In  general  there  were  three  academic  terms  of  data  collec- 
tion on  program  results  for  each  target  course. 

Nearly  200  sections  of  target  courses  were  involved  in.  the  demonstration 
period  alone.     Others  had  been  part  of  the  instrument  trials  conducted  during 
the  implementation  phase  of  the  project.     Within  each  section  the  timing  for 
administering  instruments  was  a  direct  function  of  the  instrument's  purpose. 
Pretests  along  with  questionnaires  on  students'   background  characteristics  (see 
Appendix  H)  were  given ^at  the  beginning  of  an  academic  term.     Sometimes  pretesting 
was  a  part  of  student  registration  for  target  courses  but  usually  it  wa s>M  Bfcw 
within  the  first  week  of  classes*     Likewise  posttesting  occurred  during  the  last 
week  of  regular  classes  as  well  as  the  final  exam  period.     However,  students 
working  on  the  TICCIT  system  generally  took  their  poisttest  whenever  they  completed 

\ 

their  course  work  rather  than  when  the  term  ended.     Pretesting  and  posttesting 
sessions  were  conducted  under  faculty  supervision.     Representatives  of  the 
evaluator  supervised  classes  as  students  responded  to  the  attitude  surveys • 
Then  there  were  topical  tests  administered  by  instructors  as  math  students 
finished  their  study  of  certain  units  within  a  course,  and  summaries  of  student 
progress  in  the  TICCIT  program  collected  automatically  by  the  computer  system. 

All  of  this  testing  and  surveying  amounted  to  an  extensive  data  base  on 
each  student's  performance.     tfven  without  the  results  of  our  observations',  which 
were  intended  as  supplementary  descriptive   information  on  class  and  student 
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activity,  there  were  enough  variables  to  get  an  accurate  and  detailed  account  of 
.program  impact.     Not  all  of   the  over  5,000  students  enrolled  in  sections  of 
target  courses  provided  data  for  the  evaluation:     some  withdrew  without  attending 
^cla^Ies,  others  failed  to  satisfy  course  requirements  within  an  academic  tferm, 
and  still  others  were  in  the  few^^t^&A where  data  collection  was  disrupted. 
But  nearly  every  student  who  started  a  target  course  became  a  part  of  our  data 
base . 

A  summary  of  the  data  base  accumulated  for  students  appears  in  Figure  2.7. 
Complete  information  was  not  available  for  each  student.     It  depended  on  whether 
he  enrolled  for  a  math  or  an  English  course,  whether  his  section  was  a  classroom 
format  or  one  on  the  TICCIT  system,  and  most  of  all  whether  or  not  he  completed 
his  course  studies.     At   least  for  those  variables  that  pertain  to  student 
characteristics   (e.g.,  age  or  sex)   it  also  depended  on  whether  a  student  objected 
to  answering  questions  of  a  personal  nature.     Obviously  we  had  to  rely  on 
techniques  of  data  anaLysis  to  make  results  clear  from  such  a  mass  of  variables. 

2.4    Data  Analysis 

The  approach  we   favor  for  data  analysis  differs  somewhat  from  traditional 
methods.     That   is  because  both  our  view  of  program  effects  and  our  preference 
for  .reporting  such  effects  follow  slightly  different   lines  of   thought.  The 
procedures  themselves  and   the  mathematical  model  underlying  them,  however, 
foLLow  well  known  and  accepted  strategies   for  data  anaLysis.      It    is   in  the 
interpretation  of   effects  that   the  key  difference  lies. 

We   tend   to  perceive  program  el  1  eel s  as  a  difference.     This  should  be  obvious 
from  our  discussion  of  comparison  groups*  and  the  care   taken    in  developing  fair 
measures  oi    effects.      lint   what    does   a  difference  between  those  outcomes  obtained 
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Data  Base  on  Student  Performance 
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Extra-curricular  Work  (none,  part-time  or 
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Prior  Enrollment  in  Course 
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Predisposition  toward  CAT  [T1CC1TJ 
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i  Program  Results 

Course  Completion  (grade  with  credit) 

Posttest  [Math] 

Score  on  Common  Objectives 

Ability  Subscores 

Content  Subscores 

Essay  Posttest  [English] 

Objective  Posttest  of  Writing  Skills 
[English] 

Topical  Tests  [Math] 

Retention  Tests  [Math] 

Attitude  Survey 

Score  on  Parallel  Statements 
Subscore  for  Personal  Attention 
Subscore  for  Satisfaction  with  Course 
Subscore  on  System  Features 

Activity  Questionnaire 

Number  of  Lessons  Passed  [T1CCIT] 

Time  on  Computer  System  [TICCIT] 
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with  the  T1CC1T  program  and  those  with  familiar  classroom  practices  mean?  Of 
course  a  significant  difference  is  evidence  of  a  program's  effect.     This  seems* 
only  to  suggest  another  question:     How  much  of  a  difference  need  there  be  before 
it  becomes  an  effect?     Let  us  begin  by  considering  what  an  effect  is  so  that  we 
can  recognize  one  when  it  occurs. 

Rubin  (1974)  defines  a  causal  effect  as  the  difference  between  what  actually 
happened  to  a  subject  exposed  to  some  treatment  and  what  would  have  happened  if 
the   same  subject  had  been  exposed   to  some  other   treatment.     He  gives,  as  an 
example,   "If  an  hour  ago  I  had  taken  two  aspirins  instead  of  just  a  glass  of 
'water,   my  headache  would  now  be  gone'1   (p.  ,689).     Thus   the  critical   (but  un- 
measurable)   quantity  of   interest   is,    tor  each  subject,   the  different  between 
the  value  of  an  outcome  under  one  condition  and   the  value  of   the  outcome  under 
another  condition.     In  other  words,  what  relative  benefit  or  detriment  might  be 
expected  at  the  end  of   the  academic   term  if  we  assign  a  student   to  a  cldss  on 
the  TICCIT  system   instead  ot   a  regular  class  section? 

In  a  sense   this  question  points   toward  an  unmeasurable  effect:     the  difference 
in  program  outcomes  toi    the   same  student.      It    is  sufficient,    for   the  moment,  to 
note  that   this  approach  places  emphasis  on  the  dllierence   in  outcome  measures 
between  programs.      And  our   on. .lysis  .strategy    leads   to  a  direct  estimate  of  this 
difference.     The  etleets  which  we   shall    report    will   he   these   estimates:     how  much 
ot   a  ditterence  the  TICCIT  program  ma<<e   in   terms  ot    items  on  an  ach  i  evement 
posttesl,   proportion  ol  '  s  t  iident  s  completing  a  course,  and  extent   oi  satisfaction 
with  a  com  He . 

Certainly  other  appro. iches   to  data  analysis  miRhl    he  advocated  with  equal 
convict  ion.      It    is  not    ..in    |un  hen'  i.>    i  ev  I  ew   their    relative  merits  or 

even  contrast    alternatives   with  .nit    approach.      However,    it    would   he   a  Kross 


misinterpretation  to  view  data  presented  in  this  report,  specifically  achieve- 
ment  data,  as  gain  scores  between  pretest  and  posttest.     Achievement  tests  were 
developed  as  measures  of  general  background  preparation  and  specific  course 
learning,  and  their  difference  does  not  reflect  gain.    Given ^jthis  caution,  we 
wish  only  to  explain  the  analysis  strategy  pursued  for  our  evaluation  in  what 

follows.  s 

2. A. a  Assumptions 

The  general  approach  to  the  analysis  consisted  of  estimating  the 
parameters  of  a  model  as  well  as  their  standard  errors  and  testing  the  statistica 
difference  between  appropriate  parameter  estimates.     In  order  to  specify  the 
mathematical  model  that  would  allow  such  estimates  and  tests,  we  need  to  make 
several  assumptions.     Perhaps  first  we  should  understand  that  there  is  a  specific 
population  on  which  we  base  all  our  estimates  and  tests.     This  population  (P)  is 
the  student  body  at  the  two  demonstration  sites,  and,  more  specifically,  those 
students  enrolled  in  target  courses  for  the  TICCIT  project.     To  the  extent  that 
these  community  college  students  resemble  other  populations,  we  can  make  general- 
izations to  other  contexts.     But  it  was  the  students  enrolled  in  algebra  and 
composition  courses  at  Alexandria  and  Phoenix  from  whom  we  collected  data  and 
for  whom  we  estimated  the  effects  of  the  TICCIT  program. 

Usually  there  is  a  well-defined  experimental  treatment  and  another  that 
serves  as  a  relative  standard  for  comparisons  or  a  control  condition.     In  our 
case  students  took  courses  under  the  TICCIT  program  (T)  or  in  a  regular  classroom 
(C).     These  laere  well  defined  instructional  conditions  in  the  sense  that  the 
TICCIT  program  has  a  consistent  teaching  strategy,  a  fixed  library  of  course 
materials,  and  a  system  designed  to  support  the  specific  strate^and  materials. 
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A  regular  classroom  relies  more  on  implicit  criteria  for  its  definition:  there 
is  an  instructor  and,  most  likely,  a  required  textbook.    What  takes  place  in  a 
classroom  (whether  primarily  lectures  or  discussions  or  reviews  of  assignments) 
and  how  an  instructor  chooses  to  teach  his  students  may  be  difficult  to  capture 
in  an  evaluation.     But  we  can  assume  such  classroom  practices  remain  fairly 
constant.  "Together  regular  classes  closely  resemble  the  practices  that  were 
followed  in  the  previous  semester  and  those  that  will  continue  in  the  next 
semester,  and  perhaps  even  those  at  another  institution.     Thus,  we  assumed  .that  T 
and  C  represented  well-defined  conditions  but  also  undertook  observations  to 

identify  exceptions. 

We  expect  an  effect,  if  it  occurs,  to  be  evident  from  an  outcome  measure 
'  dependent  on  the  treatments.     Such  a  dependent  variable  (Y)  might  be  a  measure^ 
of  course  completion,  student  achievement,  or  student  attitudes.     There  is 
further  a  definite  time  interval  of  students'  exposure  to  an  inst^tional 
condition;     it  starts  at  time  t]_  and  ends  at  t^     For  course  completion  the 
natural  time  frame  wAuld  be  the  academic  term.     So  students  would  register  for  a 
course  under  the  T  or  C  condition  at  ^  and  we  would  observe  Y,  completion 
status,  at  t2.    We  may  wish  to  impose  a  fixed  time  constraint  on  achievement 
and  attitudes  as  well  as  completion.     But  such  a  rigid  interpretation  of  exposure 
defeats  the  purpose  of  the  TICCIT  program  and,  in  effect,  forces  an  individualized 
curriculum  back  into  a  lock  step  pattern  with  an  academic  calendar.  Instead, 
whenever  a  student  completed  his  studies  on  the  TICCIT  system  was  the  time  at 
which  we  observed  the  value  of  Y  for  his  course  achievement  or  attitudes.  There 
was  an  exception  to  this  rule:     we  examined  effects  within  a  fixed  time  period 
for  the  math  courses  at  one  college.     But  this  exception  was  a  matter  of  necessity 
(see  Chapter  3:     Implementation  of  the  TICCIT  Program)  rather  than  preference. 
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Estimating  TIuCIT's  effects  would  be  a  simple  matter  if  each  student  went 
through  both  the  T  and  C  conditions  from  t^  to  t^.     Then  we  could  take  the  dif- 
ference between  his  values  for  Y  and  state  the  effect.     But  it  is  impossible  for 
each  student  to  be  in  both  conditions  at  one  time.     Although  this  sometimes  happens 
in% evaluation  studies,  it  confounds  rather  than  clarifies  the  effect.     So  we  must 
settle  on  an  approximation  to  having  the  same  students  under  both  conditions,  ^he 
same  students  cannot  take  a  course  in  two  ways  but  similar  students  can. 

Our  estimates  of  program  effects  depend  on  comparisons  between  the  values 
of  Y  for  students  in  the  T  condition  with  values  of  Y  for  similar  students  in 
the  C  condition.     Similarity  here  means  that  students  were  essentially  identical 
on  a  set  of  variables  measured  at  t^.     These  variables,  always  measured  before 
the  onset  of  the  T  and  C  conditions  or  unchanged  by  the  conditions  (e.g.,  sex), 
are  covariates.     We  denote  the  vector  of  all  such  covariates  by  X.     Two  students 
with  identical  values  of  X  may  be  considered  as  similar  as  far  as  the  relevant 

data  set  goes:     these  students  are  the  same  on  every  characteristic  measured 

,/  \^ 

prior  to  the  initiation  of  the  treatment. 

It  might  be  helpful  to  enumerate  the  above  assumptions  before  discussing 
the  mathematical  model  based  on  them. 

(1)  There  is  a  population  (P)  from  which  the  students  may  be 
regarded  as  representative.  Although  the  TICCIT  demonstration 
took  place  at  a  community  college,  the  evaluation  actually  dealt 
with  several  populations.     Those  students  who  enrolled  for  a 
target  course  defined  a  population.     Further,  most  of  the 

analyses  for  student  achievement  and  attitudes  necessarily  v 

dealt  with  the  population  which  enrolled  for  and  completed  a 

course. 

(2)  There  is  a  definite  treatment  condition  (T)  as  well  as  a 
well-defined  control  condition  (C).     For  this  project  these 
conditions  were  the  TICCIT  program  and  the  regular  classroom. 
An  additional  condition,  programmed  instruction,  was  available 
as  a  second  control  condition  for  one  math  course.  Each 
student  was  in  one  of  these  conditions. 
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(3)  There  is  a  dependent  variable  (Y)  which  functions  as  a  measure 
of  the  programs'  effects.    Our  outcome  measures  covered  course 
completion,  student  achievement,  and  student  attitudes.  Each 
was  observed  sometime  after  the  initiation  of  the  T  and  C 
conditions.     The  usual  period  of  student  exposure  to  a  condition 
was  the  academic  term. 

(4)  There  is  a  vector  of  covariates  (X)  that  determines  the  similarity 
of  the  student  groups  exposed  to  the  different  treatments. 
Whenever  possible,  assignment  to  .an  Instructional  condition  was 
random  or  blind.    However,  students  also  took  pretests  and 
provided  us  with  information  on  such  background  characteristics 

as  age,  sex,  and  grade  point  average.     These  variables  observed 
prior' to  the  onset  of  the  instructional  conditions  served  as 
our  covariates. 

2.4.b    Mathematical  Model 

The  dependent  variable  Y  and  the  vector  of  covariates  X  may  be  regarded 
as  having  a  joint  distribution  over  the  population  P.     This  distribution  depends 
on  whether  the  students  represent  the  population  as  exposed  to  the  T  or  C 
condition.     Thus,  conditibnal  on  X,  Y  has  a  distribution  dependent  on  which 
condition  the  student  is  in.     Or,  simply,  the  outcome  for  similar  students  is  a 
function  of  the  mode  of  instruction  followed  in  a  student's  class.     The  average 
value  of  Y  for  students  from  P  with  covariate  values  X  -  x  and  who  receive  the 
treatment  T  is  expressed  as 

l^(x)  -  Ep  (Y|X  -  x,  T).  .  ^ 

Similarly,  the  average  value  of  Y  for  students  from  P  who  receive  the  control 
condition  C  is  denoted  by 

Wc(x)  -  Ep  (Y|X  =  x,  C). 


The  functions  and  Wr(x),  sometimes  called  response  functions,  represent 

the  average  value  of  Y  in  the  two  conditions  for  students  from  P  with  identical 
covariate  values  x.  This  is  just  a  formal  way  to  say  that  the  outcome,  whether 
course  completion,  achievement  or  attitudes,  depends  on  the  instructional 
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condition  (T  or  C)  once  we  control  for  other  variables  (X)  thought  to  influence 
that  outcome  (e.g.,  pretest  score). 

It  follows,  then,  that  the  difference  between  these  two  functions  is  the 
treatment  effect. 

■       t(x)  -  Uj,(x)  -  yc(x)  * 
This  is  the  average  difference  in  the  value  of  Y  for  students  whom  we 
regard  as  identical  since  their  covariate  values  equal  x.     The  new  function  T(x) 
Ls  the  effect  at  x:     it  represents  the  difference  in  anticipated  program  results 
for  similar  students.     If  t(x)  is  averaged  over  the  population  P  we  obtain  the 
treatment  effect  in  the  population.     This  is  given  as 

Tp   -  Ep  [T(X)]. 

The  purpose  of  our  analysis  is  to  estimate  the  treatment  effect  Tp  for 
each  dependent  variable  Y  and  on  the  population  P  affected  by xthe  treatment. 
Our  estimate  of  t    will  be  denoted  by  f  .     If  we  further  use  a,  to 
represent  the  standard  error  of  the  average  effect,  then         denotes  the 
estimated  standard  error  for  the  distribution  Tp.     This  re^t  gives  both  the  ' 
value  of  t    aria  6\  .  1 

2,4. c    Parameter  Estimation 

The  analysis  consists  of  three  stages  of  parameter  estimation.     In  the 

first  we  estimate  the  response  functions  yT(x)  and  yc(x).     Next  we  can  use  these 

forms  to  arrive  at  the  treatment  effect  at  x,  f (x) ,  and  average  these  to  get  the 

overall  treatment  effect  fp.     Then  we  need  9f    to  test  the  statistical 

P 

significance  of  the  effect.  In  order  to  estimate  xp  we  must  start  with  uT(x)  and 
yc(x).  We  must  decide  on  a  form  for  these  response  functions  in  each  population. 
This  involves  exploring  a  number  of  possible  forms.       It  is  not  necessary  that 


ERIC 


the  two  functions  be  linear  or  parallel.    Of  course  should  the  data  support  such 
simplifications,  we  take  advantage  of  them.    Our  primar^objectiye^s  a  powerful;, 
rather  than  a  simple  form.    We  want  response  functions  which  account  for  aB  much  ; 
of  the  variance  in  Y  as  possible  given  the  available  data.    As  the  explanation 
of  Y  variance  improves,  the  accuracy  of  the  response  functions  u^(x)  and  % 
Mc(x)  also  improves.    With  better  accuracy  in  expressing  the  average  value,  of  Y 
we  also  get  fewer  alternative  explanations  for  the  difference  t(x)  and  such  a 
difference  becomes  increasingly  attributable  to  a- true  treatment  effect.  When 
response  functions  leave  much  of  the  Y  variance  unexplained,  the  difference  x(x) 
contributes  little  and  .tp  can  deg^erate  "into  ^simple-  comparison  of  the  mean 
values  for  Y  by  treatment.    When  .espouse  f u*c tions  explain  mo*  of  the  variance 
...  in  Y,  then  even!  a  relatively  small  treatment  effect  ^  ie, detected  and  reported 


1  .      '.  with  .confidence     So'  this  phase  of  estimating  Tp  is  #portant  though  exploratory. 
^  ;,  ,;We  *eek  to  reexpre  A  varies  '(e^./A.  log  x  rather  tfiaf  x)  ,  identify 
•    -Tellers  (e..g.,.  isolate,  the^lassgive^  only  half-  as  mW  time  asters  for 
V    ^eom^eting  the  potest),  and  gener^ly  bring  tools  f ro? explora^ry  data 

AjLfsl*  *?  bear  f.  the  "problem  of  simplifying  and  estimating  the 'value  of  Y  . 
A'fetvtheV  and  C  conditions.  |^        'j    -    -  ,£  [ 

P^tima<ing  r,sp|nse  Wtiofs.  can  benefit)  t*A  poolin^data  across  groups 
within  each  poiuU^'  FoFexample,  if  we  pool  data  from  day  classes  and 
eveLg  Wsses  the  size  of  4fr  data  base  increases.     Similar  increments  can 
,*esuU  from  poking  across  ac^mic  *erms  eleven  across * the  T  aXc  conditions. 
•The  only  fil/ule  'here  is  that,  we ]  only"  pool  across  groups  by  assuming  some 


version  oi 


Lel^ris^s^fYctions  in  th^  groups.    Naturally  we  cannot 


agfeate^^llar  d^ta  aTwe  would  if  we  lr|e|\p^popl  those  populations 
A"  '  '   Lfferent  courses.     Instead  we  can  consider  each  course  as  a  distinct 


•enrolled  in  dif 

A 


population  and  therefore  a  replication  trial.    The  reason  for  pooling  is  to 
increase  the  relative  sample  size,  and  thereby  improve  our  estimate  of  *  . 

The  estimated  response  functions  0T(x)  and  determine  the  treatment 

effect  at  x: 

t(x)  -  ft^x)  -  0c(x) 
This  is  our  best  estimate  of  the  relative  increase  (or  decrease)  in  Y  due  to  T 
for  students  with  covariate  values  of  x.     Now  we  can  use  the  data  to  arrive  at 
an  estimate  of  the  overall  effect  t  . 

In  the  simplest  case  we  compute  ^(x^  for  every  observation  i  in  the 
study  and  average  these  student  effects.     But  where  we  lack  complete  data 
(e.g.,  unobserved  posttest  performance  under  one  condition  due  to  a  different 
course  completion  rate)/we  can  only  estimate  Tp  averaged  over  those  cases 
for  which  X  equals  x  under  both  conditions.     The  unmatched  observations  still 
contribute  to  the  previous  phase  in  which  we  use  all  available  data  to  estimate 
the  form  of  the  response  functions.     However,  the  estimated  treatment  effect 
Tp  exists  only  where  X  equals  x  under  T  and  C. 

We  can  define  the  anticipated  difference  between  program  outcomes  as 

i   n  i   n  r 

*P  =  n  ±h  *<xi>  =  n  ih    [VV  "  0C(Xi>] 
under  T  and  C  for  n  students.     Often  the  size  of  the  data  set,  n,  is  less  than 

the  total  enrollment  for  a  course,  N.     There  will  be  students  who  fail  to  take 

the  pretest.     Others  will  not  complete  their  studies  and  thus  the  value  of  Y  for 

achievement  or  attitudes  will  not  be  observed...     Consider  the  analysis  for 

student  achievement.     The  estimated  treatment  effect  has  meaning  only  for  those 

students  who  complete  a  course  and  not  for  all  who  enroll.     Should  there  be  a 

vastly  different  completion  rate  in  one  condition  than  in  the  other  (but  still 

with  parallel  response  functions),  the  achievement  effect  actually  occurs  for  a 
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limited  number  of  cases  in  which  X  equals  x  under  both  conditions.     Even  worse 
cases  can  arise:     there  may  be  a  treatment  effect  dependent  on  the  value  of  X. 
If  this  happens  for  Y,  the  differential  effect  itself  constitutes  the  major 
finding  since  other  comparisons  of  program  outcomes  may  subsume  bias.  These 
restrictions  on  the  interpretation  of  treatment  effects  should  be  kept  in  mind 
as  cautions  against  overly  broad  judgments  of  results.     Our  average  treatment 
effect  applies  to  n  students  out  of  a  course  enrollment  of  N.     As  n  approaches 
N,  we  can  be  confident  that  the  treatment  effect,  whatever  it  might  be,  applies 
to  the  entire  population  of  students  in  a  course. 

How  do  we  know  whether  a  treatment  effect  once  estimated  constitutes  a 
difference  of  statistical  significance?    We  estimate  the  standard  error  of  the 
average  effect  a,  .     Since  the  response  functions  0T(x)  and  VQW  them- 
selves correspond'to  estimates  derived  from  regression  analysis,  the  estimated 
error  6f    comes  from  values  already  available.     We  compute  it  from  the 
standarl'error  of  estimate  made  in  fitting  a  form  to  Y,  the  number  of  covariate 
values  that  go  into  each  response  function,  and  the  number  of  observations 
averaged  for  tp.     The  computation  of  9f     can  also  depend  on  the  functional 
forms  of   0T(x)  and  tQW  and  the  placement  of  covariate  values.     In  such 
cases  we  also  need  the  covariance  matrix  for  response  functions  but  computation 
remains  a  technical  detail. 

2.4.d  Application 

Certainly  there  must  be  specific  values  associated  with  the  vector  of 
covariates  X  in  order  to  estimate  parameters  and  thereby  determine  the  extent 
and  significance  of  the  treatment  effect.     It  should  be  clear  first  that  the 
covariates  pertain  to  some  dependent  variable  Y  as  well  as  a  specific  population 
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;:       P.    What  follows  is  a  brief  discussion  of  Y,  P  and  X,  as  well  as  their  roles  in 
parameter  estimation. 

The  major  dependent  variables  in  this  study  will  be  course  completion, 
student  achievement,  and  student  attitudes.     Course,. cd^ption  is  a  simple 
dichotomous  variable:    Y±  has  a  value  of  1  if  student  i^ceives  college  credit 
for  his  course  work  and  0  if  he  does  not.     Where  the  dependent  variable  is 
achievement,  Y±  is  the  number  of  posttest  items  to  which  student  i  gave 
correct  answers.     Actually  this  may  just  reflect  those  items  on  objectives 
common  across  conditions  for  math  courses,  or  simply  a  score  on  a  four-point 
scale  that  instructors  assigned  to  essays  written  as  English  posttests.  An 
attitude  score  as  the  dependent  variable  represents  the  extent  of  a  student's 
favorable  reaction  to  his  course.     It  is  the  sum  of  a  student's  responses  to 
approximately  twenty  statements,  with  each  response  on  a  five-point  scale  of 
agreement. 

The  populations  essentially  correspond  to  the  eight  courses  (see  Figure  2.4) 
for  which  the  TICCIT  program  was  a  curricular  alternative  to  regular  classes. 
Our  evaluation  deals  with  program  results  for  each  of  the  eight  courses  in  at 
least  two  and,  in  most  cases,  three  academic  terms.     Together  with  the  preliminary 
trials  of  the  implementation  period  for  the  TICCIT  project,  we  can  track  results 
across  as  many  as  six  terms.     The  product  of  the  number  of  courses  and  number  of 
terms  gives  the  number  of  populations  at  one  extreme.     But  the  demonstration 
really  involves  only  eight  populations  if  we  view  the  separate  terms  as  repli- 
cation trials.     At  another  extreme  would  be  one  population  if  we  accepted  the 
TICCIT  program  as  a  general  currtcular  strategy  independent  of  college,  course, 
and  term.     Our  course-specific  measures  of  achievement  results  preclude  such  an 
extreme  position.    We  will  conduct  our  analyses  by  course  and  term,  and  collapse 
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.cross  terms  to  attain  larger  sample  sizes.     Of  course  the  population  also 
depends  on  the  nature  of  the  dependent  variable:     achievement  and  attitude 
results  generally  pertain  to  those  students  who  complete  a  course  and  not  who 
enroll.     So  there  will  be  subpopulations  in  each  of  the  eight  courses  defined  by 
the  dependent  variables. 

The  covariates  in  our  analyses  fall  into  three  categories.     The  first  and 
most  important  is  the  pretest.     It  serves  as  a  measure  of  student  ability  and 
preparation  prior  to  the  start  of  a  course  and  usually  accounts  for  a  good  deal 
of  the  variation  in  posttest  performance.     For  math  we  will  use  one  such  pretest 
per  course,  while  for  composition  courses  we  take  both  an  essay  and  an  objective 
test  as  entrance  measures.     We  will  still  refer  to  only  one  pretest  score,   S,  for 
each  student  although  two  enter  into  the  response  functions, for  composition 
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A  second  category  of  covariates  relates  to  the  demographic  characteristics 
of  the  student.     Some  of  these  characteristics  will  be  simple  dichotomous 
indicators  while  others  will  be  on  a  continuum.     Sex,  student  status,  and 
enrollment  history  exemplify  traits  for  which  an  indicator  with  values  of  1  or  0 
suffices.     A  value  of   1  might  represent  male,  full-time  student,  or  a  repeat 
enrollment  for  a  specific  course,  and  a  value  of  0  female,  part-time  student,  or 
first  course  enrollment.     We  need  one  dichotomous  variable  for  each  of  these 
traits.     Generally,  R  will  stand  for  the  vector  of  j  such  covariates  where  each 
student  has  supplied  values  for  all  j  characteristics.     Demographic  variables  on 
a  continuum  might  be  age,  number  of  college  credits,  college  grade  point  average, 
high  school  grade  point  average,  and  other  characteristics  on  an  interval  scale. 
We  will  denote  the  collection  of  these  k  characteristics  by  the  covariate  vector 
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Q.    Again,  there  must  be  observations  on  all  k  such  variables  for  each  student 
in  order  for  them  to  be  useful  In  estimating  response  functions. 

Another  class  of  covariates  applies  to  groups  of  students  rather  than 
each  individual.     These  categorical  variables  merely  distinguish  between  types 
of  sections,  for  example  those  in  the  college's  day  division  and  those  in  the 
evening,  or  those  meeting  three  periods  per  week  and  those  meeting  five  periods 
for  the  same  course.     In  analyses  across  academic  terms  there  would  be  a  group 
variable  for  classes  that  met  in  the  fall  term  as  contrasted  to  those  in  the 
spring.     These  variables  permit  contrasts  to  be  made  among  natural  subpopula- 
tions.     Where  no  evidence  suggests  a  treatment  effect  dependent  on  such  groups, 
we  can  pool  data  across  these  subpopulations .     Otherwise  we  must  estimate  the 
differential  treatment  effects  for  the  specific  subpopulations.     Whether  we 
proceed  to  estimate  one  or  several  treatment  effects  depends  on  the  form 'of 
the  response  functions  established  in  the  first,  exploratory  stage  of  parameter 
estimation.     We  will  use  the  covariate  vector  M  to  represent  the  h  group  vari- 
ables.    Like  the  vector  R  for  dichotomous  student  characteristics,  variables 
indicated  through  M  usually  assume  one  of  only  two  values  (e.g.,   I  for  day  and  0 
for  evening).     Unlike  either  R  or  Q,  two  students  in  the  same  class  must  have 
identical  M  values  since  it  is  a  group,  not  an  individual,  vector.     It  might  be 

considered  as  a  shorthand  notation  for  blocking  factors  or  grouping  contrasts. 

* 

There  is  a  fourth  category  of  covariates  that  does  not  enter  into  all 
analyses:     variables  that  indicate  instructors.     Indicator  variables  for 
instructor  appear  only  in  the  English  courses.     The  TICCIT  program  for  math  is 
independent  of  instructors  to  the  extent  that  one  college  had  no  course  sections 
as  such  and  instead. let  students  choose  their  own  time  schedule  on  the  computer 
system  and  instructors  rotate  their,  time  schedule  as  resource  teachers.  The 
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design  of  the  TICCIT  program  did  call  for  a  stand-alone  approach  to  computer- 
assisted  instruction.     But  one  college  decided  to  use  the  program  as  an  adjunctive 
resource  in  its  composition  course.     We  Report  this  fact  now  because  it  bears  on 
the  analysis  strategy  and  defer  further  explanation  until  the  next  chapter.  An 
instructor  might  teach  under  only  the  T  condition  or  only  the  C  condition,  or 
both  the  T  and  C  conditions.     It  is  when  an  instructor  teaches  both  ways  that  we 
can  estimate  dependence  of  the  response  functions  on  that  instructor.  Otherwise 
the  variation  in  Y  that  might  be  attributed  to  instructor  is  confounded  with 

treatment.     Let  1  denote  this  vector  of  g  ins tructors ,  then 

!1   if  student  i  has  instructor  g 
0  otherwise 

This  vector  serves  a  purpose  similar  to  that  of  the  group  vector.     It  blocks 
student  data  according  to  instructor. 

For  a  given  population  P  there  is  now  covariate  information  about 

S    pretest  score 

R    demographic  characteristics  on  a  noAjfell  scale  (j  variables) 
Q    demographic  characteristics  on  an  ij^fflpl  scale  (k  variables) 
M    group  factors   (h  variables)  * 
I     instructor  indicators   (g  variables) 

and  the  response  functions  ray  be  restated  as 

HT  (S,   {Rj},   <  <v  *     1MW>,   {Ig})  and 

pc  (S,   {R.},  {Qk>,  <\),  Ug>) 

where  I     only  enters  into  the  English  analysis.     The  regression  analogues  to 
g 

simplified  response  functions  would  take  the  form: 

MT(X)  =  b    +  bl  (S)  .+  E  c    Rj  +.1  dk  Qk  +  E  eh  ^  +  E  f    I    +  u 
MC(X)  =  bQ  +  bj   (S)  +  z  c.  Rj  +  E  dk  Qk  +  I  eh  ^  +  E  fg  Ig 
These  simplified  forms  assume  that  the  slopes  on  S  are  identical  across  groups 
and  conditions.     This  is  a  testable  hypothesis' through  interactions  of  S  with  M 
and  T  or  C.     Where  exploratory  analysis  for  the  response  functions  contradict  the 
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hypothesis  of  identical  slopes,  we  will  examine  more  complicated  forms  of  the 
response  functions. 

We  actually  estimate  response  functions  in  regressions  of  the  dependent 

variable  Y  on  the  independent  covariates  S,  <R  },  <o  },  {MK  and  {I  }  with 

j  K         h  g 

one  additional  variable  TICCIT,  where 
1  if  student  i  is  in  T 


TICCIT    -  .     j  c 

i      ^0  if  student  i  is  in  C. 

If  exploratory  analysis  justifies  the  simple  model,  then  the  estimated 

treatment  effect  fp  is  simply  u.     The  regression  reports  its  value  as  the 

coefficient  for  our  variable  TICCIT,  and  its  standard  error  <Jf     follows  from 

P 

the  standard  error  for  the  regression  weight  on  TICCIT.     Where  it  becomes 
necessary  to  reject  the  simplified  regression  model  due  to  interactions,  there 
will  be  an  explanation  of  the  form  found  for  the  response  functions.     It  may  be 
a  substantive  reason  that  has  policy  implications  for  the  program  or,  more 
likely,  an  effect  of  maturation  or  another  internal  source  of  invalidity. 

Of  course  not  all  our  analyses  follow  these  procedures  for  data  analysis. 
We  do  adhere  to  the  procedures  as  much  as  possible  in  studying  course  completion 
student  achievement,  and  student  attitudes.     But  for  student  activities  and 
faculty  acceptance  the  analyses  tend  to  be  d^criptive  rather  than  inferential. 
And  some, cases  will  arise  where  the  number  of  covariates  exceed^  the  size  of  the 
subpopulation  involved  in  a  specific  response  function.     Reason,   too,  plays  a 
role  in  data  analysis,  as  does  knowledge  of  the  variables  and  conditions. 
Perhaps  the  context  in  which  the  evaluation  occurs,  though,  exerts  a  stronger 
influence  ^  results  than  any  of  our  careful  plans,  preparation  of  instruments, 
data  collection,  or  data  analysis. 


Chapter  3 

Implementation  of  the  TICCIT  Program 

Regardless  of  results  the  introduction  of  an  innovative  curriculum 
into  schools  is  in  itself  an  achievement.     This  is  especially  true  when  the 
curriculum  involves  an  application  of  inaxructional  technology  with  the 
stated  objective  of  supplanting  traditional  metfu^s  of  instruction  entirely. 
Cooperative  and ^recept ive  school^  need  to  be  enlisted  as  demonstration 
site^s  for  the  curriculum  program.     And  the  particular  schools  may  affect 
the  extent  to  which  an  evaluation's  results  depict  fairly  what  may  be 
expected  to  happen  elsewhere  with  other  students  or  under  different  condi- 
tions*   There  must  also  be  mechanisms  for  maintaining  the  commitment  and 
involvement  of  selected  schools  as  well  as  for  preparing  these  demonstration 
sites  for  the  use  of  the  curriculum  program.     If  evaluation  results  do 
reflect  the  real  impact  of  the  curriculum,  there  is  also  reason  to  consider 
its  dpsign  and  development^ as  they  might  bear  on  other  projects.  These 
processes  of  selecting  demonstration  sites,  preparing  schools  for  the 
adoption  of  the  curriculum,  and  developing  the  actual  curriculum  materials 
represent  components  of  the  whole  implementation  process. 

This  chapter  addresses  the  implementation  of  the  TICCIT  program. 
Unlike  subsequent  chapters  dealing  with  program  outcomes,  the  focus  here  is 
on  the  processes  involved  in  introducing  the  TICCIT  program  to  community 
colleges  and  in  developing  courseware.     The  first  section  presents  the 
rationale  behind  the  choice  of  community  colleges  as  demonstration  sites 
and  the  factors  taken  into  consideration  in  the  selection  of  particular 
colleges  as  participants  in  the  project.     It  also  describes  the  students 


enrolled  at  these  colleges  and  the  courses  offered  on  the  TICCIT  system. 

A' 
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The  second  section  covers  those  aspects  of  project  administration  expressly 
concerned  with  facilitating  the  implementation  of  the  TICCIT  program,  and 
recounts  'the  colleges'  staffing  of  the  TICCIT  program.    Finally  there  is 
an  Overview  of  the  design  and  production  of  TICCIT  courseware. 

3.1    Demonstration  Sites 

The  demonstration  of  the  TICCIT  computer-assisted  instructional  system 
took  place  in  community  colleges.    The  particular  commu^ty  colleges  chosen 
as  demonstration  sites,  their  respective  student  populations^ and  the 
'  specific, courses  offered  under  the  TICCIT  program,  undoubtedly  played  a 
maJor  role  in  determining  the  results  reported  in  this  evaluation  study. 
Also,  any  generalizations  based  on  the  evaluation  must  take  into  considera- 
tion the  extent  to  which  the  characteristics  of  these  particular  community 
colleges  conform  with  or  depart  from  the  characteristics  of  other  possible 
sites.     Both  for  appreciating  the  results  attained  in  the  TICCIT  project 
and  for  making  Terences  about  probable  outcomes  in  other  situations,  it  . 
may  be  helpful  to  examine  the  context  of  the  evaluation. 

The  decision  to  enlist  community  colleges  as  participants  in  the 
demonstration  was  a  careful  and  planned  choice  consistent  with  the  project's 
goals.     The  developers  of  the  TICCIT  program  sought  to  create  a  market 
success  for  computer-assisted  instruction &nd'  community  cd^s  repre- 
sented a  market  with  the  potential  for  high  voile  and  low  resistance. 
Among  all  sectors  of  education,  community  colleges  had  shown  the  most 
dramatic  upsurge  in  enrollment,    Their  total  student  enrollment  had  increased 
three-fold  between  1960  a|f>70,  and  it  was  expected  to  double  within 
another  decade.     Furthermore,  the/,  was  a  concentration  of  this  total 
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enrollment  in  certain  courses,  notably  introductory  courses  in  English  and 
mathematics.     If  students  took  just  one  such  course  per  academic  term  and 
if  computers  could  support  full  courses  in  these  disciplines,  the  TICCIT 
program  might  assume  20%  of  the  instructional  load  at  community  colleges. 
It  seemed  clear  that  community  colleges  presented  a  viable  market  from  the 
standpoint  af  volume. 

Estimates  of  instructional  costs  also  contributed  to  the  selection  of 
community  colleges  for  demonstrating  the  TICCIT  program.     Its  developers 
believed  that  the  costs  of  the- TICCIT  system  would  be  within  reach  for 
community  colleges  with  a  minimum  enrollment  of  800  ^^ents .  Forecasts 
indicated  that  nearly  three-fourths  of  all  community  colleges  would  be  that 
large  by  1981.     These  projections  assumed  that  the  TICCIT  program  would 
support  about  20%  of  all  student  contact  hours  at  a  college  and  thus 
justify  its  purchase  cost  to  the  college.     The  developers  found  further 
support  for  the  financial  feasibility  of  the  project  in  the  space  and 
faculty  demands  of  traditional  instruction.    A  survey  conducted  in  1968-69 
revealed  that  community  colleges  devoted  18  square  feet  of  classroom  space 
per  student,  whereas  the  individual  carrel  space  required  for  the  TICCIT 

project  would  be  only  12  square  feet  per  student.    Despite  their  rapid 

t 

growth  in  enrollment,  the  student-faculty  ratio  at  community  colleges  had 
fallen  from  25.77  to  1  in  1960-61  to  20.03  to  1  in  1969-70.     This  fact 
coupled  with  a  stpady  increase  in  average  faculty  salary  meant  that  the 

osts  of  traditional  instruction  would  continue-  to  rise.  T^^LCCIT 
program  was  to  permit  much  higher  student-faculty  ratios  through t  its  main- 
line instruction,  and  higher  ratios  Implied  greater  cost-effect iveneas . 


.4 
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Although  educational  institutions  may  tend  to  be  conservative  and 
somewhat  resistant  to  change,  community  colleges  were  thought  to  be  recep- 
tive  to  innovations  such  as  computer-assisted  instruction.    A  number  of 
colleges  had  already  adopted  learning  systems  geared  toward  the  individual 
student.    Community  colleges  generally  had  a  policy  of  open  enrollment  and, 
hence,  a  sizeable  percentage  of  students  weak  in  basic  skills.    This  made 
individualized  instructional  systems  particularly  attractive  for  community 
colleges.    Indeed,  many  state  codes  for  the  construction  of  new  junior 
colleges  recommended  learning  resource  centers  with  individual  carrels 
rather  than  the  physical  layout  of  traditional  libraries.     It  also  seemed 
that  administrators  at  community  colleges  retained  substantial  authority 
for  making  decisions  and  commitments.     Such  central  responsibility  suggested 
the  feasibility  of  reaching  timely  agreements  with  selected  demonstration 
sites  and  maintaining  channels  for  smooth  communication.     For  their  growth 
and  potential  market  size,  their  potential  for.a  cost-effective  application 
of  computer-assisted  instruction,  and  their  apparent  receptivitf  to  innova- 
tive instructional  systems,  the  developers  of  the  T1CC1T  program  chose  com- 
munity colleges  as  appropriate  demonstration  sites. 

3.1. a     Site  Selection 

The  selection  of  specific  community  colleges  to  participate  in 
the  demonstration  took  place  in  the  1971-72  academic  year.  Geographical 
proximity  to  the  developers  was  a  key  factor  in  the  selection  process. 
Project  resources  and  time  schedules  did  not  permit  an  open  search  for  ^ 
participants.     Instead  the  developers  sought  one  site  close  to  the  location 
of  system  design  and  another  site  close  to  the  center  for  courseware 
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production.    Although  demonstration  sites  located  close  to  the  developers 


V 


could  come  to  depend  heavily  on  external  support,  the  wide  geographical 
separation  of  system  design  and  courseware  production  meant  that  one  site 
would  still  be  quite  distant  from  the  system  engineers  and  another  far  from 
the  courseware  specialists.     Actually,  it  became  necessary  for  representa- 
tives of  both  courseware  production  and  system  design  to  spend  exte^tf&l 
periods  of  time  at  the  colleges.     The  installation  of  TICCIT  system^  "and 
the  refinement  of  courseware n required  heavy  external  support.     Such  depen- 
dence on  developeris  shduld  perhaps  be  expected  for  the  initial  implemen- 
tation of  a  major  instructional  program.     Criteria  beyond  geographical 
proximity  farther  narrowed  the  candidates  for  participation.  College 
facilities  had  to  accommodate  the  physical  space  needed  for  the  TICCIT 
hardware  and  terminals.     Support  staff,  such  as  computer  operators  and 
student  proctors,  had  to  be  available  at  the  colleges.     Enrollment  had  to 
be  high  enough  to  j ustif y  a . f ull  TICCIT  system  with  128  terminals,  in 
anticipation  of  sufficient  student  use  to  achieve  cost-effectiveness. 
And  there  had  to  be  substantial  overlap  between  the  college's  curriculum 
structure  and  the  courseware  planned  for  the  TICCIT  project. 

According  to  these  criteria  it  was  possible  to  identify  several 
eligible  participants.     Selection  among  these  eligible  participants  then 
focused  on  their  student  enrollment  in  courses  covered  by  the  TICCIT 
program  and  their  student  profile  in  comparison  with  national  norms  for 
community  colleges.     The  two  colleges  with  the  highest  enrollment  in  target 
courses  and  the  greatest  similarity  to  national  norms  received,  and  accepted, 
invitations  to  participate  in  the  project.     They  were  the  Alexandria  Campus 
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of  Northern  Virginia  Community  College  and  Phoenix  College  of  the  Maricopa 
County  Community  College  District.     Both  Northern  Virginia  Community  College 
and  Maricopa  County  Community  College  District  had  several  community  college 
campuses  tied  together  under  a  single  administrative  bo*rd .    These  systems 
had  been  approached  initially  due  to  their  proximity  to  either  the  MITRE 
Corpor^ion  in  McLean,  Virginia  or  Brigham  Young  University  in  Provo,  Utah. 
The  criteria  for  facilities,  support,  enrollment,  curriculum  and  representa- 
tiveness guided  the  final  selection  of  the  Alexandria  and  Phoenix  campuses. 

Agreements  between  the  developers  and  the  colleges  stipulated  the 
responsibilities  of  each  participant.     The  MITRE  Corporation  would  install 
a  TICCIT  system  at  each  demonstration  site  during  the  summer  of  1974,  and 
it  would  provide  courseware  equivalent  to  two  mathematics  courses  and  two 
English  courses.     The  developers  also  promised  zo  train  teachers  and 
deliver  manuals  for  the  use  of  the  TICCIT  system.     In  return,  the  colleges 
would  support  the  installation  and  operation  of  the  system-     Their  specific 
responsibilities  included  facility  modifications,  carrel  construction, 
utilities,  air  conditioning,  and  support  personnel  (technicians,  proctors, 
instructors,  and  a  local  project  director).     It  was  also  agreed  that  each 
demonstration  site  would  cooperate  with  an  independent  agency  in  the  conduct 
of  a  formal  evaluation.     Finally,  the  agreements  called  for  establishing  a 
committee  with  the  expressed  purpose  of  coordination  among  participants  in 
the  project.    That  committee  included  representatives  of  the  developers, 
demonstration  sites  and  the  evaluator.     Its  meetings  would  serve  as  an 
impetus  for  reviewing  the  project's  status  and  for  taking  action  to  resolve 
any  unanticipated  problems. 


v. 

 v  •  ^ 

The  process  of  (-"Selecting  'demonstration  stAes  for  the  TIGCIT  project 


jtect.  and  carried  but  Ififi  an  dStemplary  manner  •     It  succeeded  in 


had  been 

^enlis^in^i^l^  cooperation  bf^wo  comfynit^  ^colleges  which  met  the  criteria 

•  X  *\         -  .  r  \ 

" ^^^^^^^*5>'^^^^^^^^"^=J^^C^^  fc <> ^  V^^^^^"*"3C"  ^omm**me^  to  the  Pr°ject  became 

evident  J^e^  as  the  colleggsVworked  together^With  the  developers  toward 

the  comply^ion  of  the  TICCIT  program.    The  developers'   foresight  "in  inviting 

the  eval^uator  to  participate  in  tfite  selection  and  in  making  cooperation 

with  the  evaluator  part  of  the  agreemrats  reached  with  the  colleges 

also  established  the  evaluation  as  an  important  component  of  the  project. 

This  early  recognition  of  the  evaluation's  importance  set  the  foundation 

for  the  colleges'   cooperation  in  all  phases  of  this  study. 


3.1.b     Student  Profile 

Each  of  the  community  colleges  chosen  to  be  participants  in  the 
project  had  a  high  student  enrollment.    This  was  a  prerequisite  to  partici- 
pation since  the  number  of  students  in  just  a  few  selected  courses  had  to 
justify  the  use  of  a  full  TICCIT  system  with  128  terminals.     At  the  start 
of  the  demonstration  in  the  1975-76  academic/ year ,  the  Alexandria  Campus  of 
Northern  Virginia  Community  College  had  an^enrollment  of  over  9,000  students 
and  Phoenix  College  of  the  Maricopa  County  Community  College  District  had 
nearly  14,000  students  in  attendance.     Because  the  characteristics  of  these 
student  populations  may  determine  whether  generalizations  to  other  contexts 
would  be  appropriate ,  a  brief  profile  of  the  students  at  each  college 
appears  in  Table  3.1. 

Consistent  with  the  nature  of  the  student  population  at  most  community 
colleges,  the  typical  student  at  Alexandria  and  Phoenix  was  registered  on  a 

% 
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Student  Profile 
Fall  Academic  Term,  1975-76 


Alexandria  Campus 
N  % 


Phoenix  College 
N  % 


? 


Total  Student  Enrollment    9319  100.0 


Curriculum  Level 

Freshman 

Sophomore 

Unclassified 

Full-time 

Part-time 

Sex 

Male 
Female 

Age 

17 
18-21 
22-24 
25-34 
35-44 
45-54 
55-64 
65- 

Ethnic  Background 


Marital  Status 

Single 

Married 

Unknown 

Residence 

State 
Other 


9 

ERLC 


1243 
643 
7433 
2342 
'6977 


4499 
4820 


123 
2271 
1567 
3624 
932 
481 
175 
42 


5225 
3650 
394 


7810 
1509 


13.3 
6.9 
79.8 
25.1 
74.9 


48.3 
51.7 


1.3 
24.8 
17.0 
39.3 
10.1 
5.2 
1.9 
.4 


American  Indian 

27. 

3 

Black  ( 

1169 

12 

5 

Oriental  1 

219 

2 

4 

Spanish  surnamed 

173 

i 

9 

Caucasian 

7471 

80 

.2 

Other  \ 

260. 

2 

.8 

56.4 
39.4 
4.3 


83.8 
16.2 


Total  Student  Enrollment    13990  100.0 


Curriculum  Level 

Freshman 
Sophomore 

Full-time  (>12  hrs.) 
Part-time  (<12  hrs.) 


10542 
3448 


75.4 
24.6 


4314  (  30.8 
9676  69.2 


Sex 


Male 
Female 


7571 
6419 


54.1 
45.9 


Age 


5-19 

2903 

20.8 

10-25 

4116 

29.4 

26-35 

4661 

33.3 

36-45 

1416 

10.1 

46-55 

615 

4.4 

55- 

221 

1.6 

Unknown 

58 

.4 

.c  Background 

American  Indian 

214 

1.5 

Black 

500 

3.6 

Oriental 

87 

.6 

Me  xi  can-Ame  r i  c  an 

1377 

9.8 

Other 

11812 

84.4 

Marital  Status 

Single 

Married 

Unknown 

Residence 


County 

State  (not  county) 

Out-of-state 

Foreign 


7258 
5784 
948 


13192 
81 
612 
105 


51.9 
41.3 
6.8 


94.3 
.6 
4.4 
.8 


Table  3.1 
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part-time  basis.    Less  than  one-third  of  the  students  had  enrolled  for  a 
full  complement  of  courses.    The  large  percentage  of  unclassified  students 
at  Alexandria  (79.8%)  reflects  the  role  of  the  college  in  serving  the 
interests  of  the  community.     Students  often  registered  for  particular 
courses  without  any  definite  plans  for  completing  ,a  two-year  curriculum  of 
studies.     Phoenix  College  considered  such  students  to  be  freshmen.  Its 
higher  proportion  of  students  at  the  sophomore  level  (24.6%)  perhaps 
attests  to  its  academic  history  and  reputation:     Phoenix  College  had  been 
established  in  1920.     Northern  Virginia  Community  College  came  about  as  a 
result  of  legislation  passed  in  1964,  and  its  Alexandria  Campus  moved  into 
permanent  facilities  for  the  1973-74  academic  year.    Thus,  Alexandria  was  a 
much  younger  college  and  had  fewer  students  committed  to  the  pursuit 
of  a  formal  degree. 

Both  colleges  attracted  students  of  a  wide  range  of  ages.  Most 
students  were  between  25  and  35  years  of  age.     The  student  populations  also 
tended  to  reflect  the  ethnic  background  of  the  surrounding  community.  Black 
students  constituted  the  largest  minority  attending  Alexandria  while 
Mexican-Americans  weroe  the  largest'"minority  enrolled  at  Phoenix.  About 
two-fifths  of  the  students  were  married,  and  the  majority  of  the  students 
held  jobs  (see  Appendix  L).     Alexandria,  as  might  be  expected  from  its 
location  in  the  Washington,  DC  metropolitan  area,  drew  a  larger  percentage 
of  students  from  other  states  of  residence  than  did  Phoenix.     Phoenix  drew 
heavily  from  its  home  county.     Further  information  about  those  students  in 
courses  offered  on  the  TICCIT -sys tem  can  be  found  in  Appendix  K,  which 
lists  their  reasons  for  electing  a  particular  section,  and  Appendix  L, 
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which  includes  additional  demographic  characteristics.  On  the  whole,  the 

two  community  colleges  selected  as  demonstration  sites  appear  to  be 

similar  to  most  other  community  colleges  of  comparable  size. 

3.1.c     Target  Courses 

The  selection  of  courses  to  be  Implemented  on  the  TICCIT  system 
was  as  careful  and  deliberate  as  the  selection  of  demonstration  sites. 
Opportunity  had  been  a  major  consideration  in  determining  appropriate 
target  courses.     Its  developers  wanted  the  TICCIT  program  to  be  cost-effec- 
tive, and  this  meant  distributing  the  costs  of  the  computer  system  among  a 
large  number  of  students  while  keeping  the  expenses  of  curriculum  develop- 
ment to  a  minimum.     The  large  numbers  of  students  enrolled  for  just  a  few 
course's  in  introductory  algebra  and  basic  English  implied  that  the  oppor- 
tunity for  achieving  a  -ost-ef f ective  b#ance  existed  at  community  colleges. 
Enrollment  figures  for  remedial  courses  in  mathematics  and  English  could 
also  be  taken  as  evidence  of  a  need  for  an  innovative  approach  to  instruction. 
The  TICCIT  program  offered  content  coverage  parallel  to  those  community  college 
courses  in  greatest  demand,   and  it  provided  an  instructional  system  for 
promoting mastery  among  individual  students. 

,  jjHfey^h^irathematics  curriculum  available  on  the  TICCIT  system 
(sX^&\  A)  corresponds  to  a  sequence  of  three  separate  algebra 
.courses,  and  t>e  English  courseware  (see  Appendix  B)   is  roughly  equivalent 
to  one  course  i\^undamentals  of  the  English  language  and  another  in 
composition.     Thus, "a  student  enrolled  for  four  semesters  of  academic  study 
might  take  one  course  per  term  on  the  TICCIT  system.     Through  a  limited 
number  of  carefully  chosen  courses  the  TICCIT  program  could  support  as  much 
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as  one-fifth  of  the  courses  a  community  college  student.     But  the 

colleges  first  had  to  approve  courses  taken  on  the  TICCIT  system  as  fully 
acceptable  alternatives  to  courses  taught  in  a  conventional  manner. 

The  colleges  themselves  decided  which  mathematics  and  English  courses 
might  best  fit  the  curriculum  available  on  the  TICCIT  system.     For  this 
reason  the  content  specifications  of  *the  courseware  adhered  closely  to  a 
traditional  structure  for  subject  matter.     It  was  in  delivery  and  design 
that  the  TICCIT  program  afforded  leverage  tor  instruction.  Courseware 
coverage  was   intended  to  facilitate  a  smooth  integration  into  the  existing 
academic  curr iculunr.     The  mathematics  faculty  at  Phoenix  College  adapted 
the  courseware  to  fit  their  most  popular  sequence  of  algebra  courses  while 
the  mathematics  department  at  the  Alexandria  Campus  of  NVCC  concentrated  on 
use  of  the  TICCIT  program  for  devel oriental  courses  in  algebra.  Conversely, 
members  of  the  English  department  at  Phoenix  initially  adopted  the  TICCIT 
program  for  support  of  their  remedial  course  in  English  fundamentals  while 
.faculty  at  Alexandria  applied  the  courseware  for  their  course  in  composition 

Textbooks  and  Requirements.     The  four  mathematics  courses  given  on  the 
TICCIT  system  at  Phoenix  College  spanned  a  full  sequence  of  algebra  courses. 
Beginning  Algebra  covered  rational  expressions,  linear  equations  and 
polynomial  expressions  with  an  introduction  to  sets.     Students  in  sections 
of  Beginning  Algebra,   listed  as  Math  007  throughout  this  report,  had  to 
complete  units  22,  20  and   17  on  the  TICCIT  system  in  order  to  receive 

credit  for  the  course.     Students  in  lecture  sections  followed  Keedy  and 
Bettinger's   Introductory  Algebra*     Tt  was  expected  ^that  students  in  lecture 
sections  would  meet  for  three- class  periods  each  week  for  the  duration  of 
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the  semester  and  would  receive  homework  assignments.     It  was  recommended 
that  students  in  TICCIT  classes  schedule  six  hours  per  week  on  the  computer 
system  and  continue  attending  these  classes  until  they  completed  the  TICCIT 
units  necessary  for  course  credit.     All  students  enrolled  for  Math  007  took 
tt\e  same  final  examination,  which  was  also  the  posttest  for  the  evaluation. 

Two  other  courses  represented  the  continuation  of  the  algebra  sequence 
at  Phoenix.     Intermediate  Algebra  and  College  Algebra,  Math  106  and  Math 
117  respectively,  also  carried  three  credits  and  met  for  three  classroom 
sessions  per  week  in  the  lecture  format.     Their  content  coverage,  in  terms 
of  the  textbooks  required  in  regular  classes  and  the  courseware  units 
required  in  TICCIT  sections,  is  given  in  Table  3.2,     The  fourth  math  course 
offered  on  the  TICCIT  system  was  really  a  combination  of  Beginning  Algebra 
and  Intermediate  Algebra.     Called  Fundamentals  of  Algebra,  it  met  for  five 

/ 

classroom  periods  per  week  as  a  lecture  class  ancj-^carr ied  five  credits. 
There  was  a  common  final  examination,  which  also  served  as  the  posttest  for 
the  evaluation,  for  all  sections  of  each  of  these  algebra  courses  (see 
Appendix  C)  . 

At  Alexandria  the  two  mathematics  courses  offered  on  the  TICCIT  system 
were  part  of  a  developmental  math  sequence.     Their  credits  could  not  be 
applied  toward  transfer   to  a  tour-year  college.    .Both  Algebra  I  and  Algebra 
II  had  five  class  sessions  per  week  throughout   the  quarter.     Students  who 
registered  for  lecture  sections,  programmed  instruction,  or  TICCIT  classes, 

L 

had  to  attend  classes  for  five  periods  per  week.     Rather  than  a  point  scale 
or  usual   letter  grades,   students  completing  all  of  the  required  course 
material  received  a  "satisfactory11  grade.     Taking  a  posttest  for  the  purpose 
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of  this  evaluation  was  also  made  a  prerequisite  to  successful  completion 
of  the  courses.     The  textbooks  adopted  for  Algebra  I  and  Algebra  11  were1 
Russell  and  Collins'  Elementary  Algebra  and  Russell  an^Lanning's  Inter- 
mediate Algebra,  respectively.     The  courseware  units  required  in  T1CC1T 
classes  appear  in  Table  3.2   (see  also  Figure  1.3  and  Appendix  A  for  unit 
contents) • 

The  course  in  English  fundamentals  given  at^ Phoenix  was  intended  for 
students  who  needed  remedial  instruction  prior  to  enrolling  in  a  composition 
course.     English   19  and  English  29  provided  a  review  of  basic  skills  and 
concentrated  particularly  on  sentence  and  paragraph  structure.  Essentially 
the  two  courses  were. parallel  in  emphasis  and  coverage  and  just  had  different 
designations  in  the  day  and  evening  divisions  of  the  college.     Both  met,  for 
three  class  periods  per  week  throughout  the  semester.     Instructors  for 
lecture-discussion  sections  tended  to  use  the  same  textbooks  in  the  day  and 
evening  divisions,  and  students  in  TICCIT/classes  had  to  complete  the  same 
map  of  courseware  units.     That  map  closely  followed  the  suggested  courseware 
structure  (shown  in  Fi^iire  1.4)  except   that  it  omitted  units  pertinent  to 
organizing  and  writing  essays   (units  9  and  1). 

The  faculty  of  the  English  department  at  Alexandria  chose  to  use  the 
TICCIT  system   In  an  adjunctive  manner  for  their  course   in  composition.  s 
Rather  than  the  usual   three  classroom  sessions  per  week,  students  in  TICCIT 
classes  went  to  work  on  the  computer  system  for  those   session^  specified  by 
their    Instructor.     The  student   time  spent  on  the  TICCIT  sys-tem  was  in  pl^jce 
of  regular  classroom  periods.     Which  courseware  units  a  student  had  to 
study  depended  both  on  the  class  assignment  s  made  by  a  particular   inst rue  tor 


Academic 

College       Term  Course 

Alexandria  Quarter    Hath  31  Algebra  I 

Hath  32  Algebra  II 


(1975-76)  English  HI  English 
Jomposltlon  I 

(1976-77)  English  111  English 
Composition  I 

Phoenix    A  Semester    Hath  007  Beginning 
Algebra 

Hath  106  Intermediate 
Algebra 

Hath  10b  fundamentals 
of  Algebra 

Math  11?  College  Algebra 


English  11  and  English  29 
Basic  English 


Number  of         Number  of 
Credits        Class  Periods 


Textbooks 


TlCClT 
Courseware 


5  5  per  week      Russell  &  Collins       Units  22,  21,  20,  M 

(non-transfer)  Elementary  Algebra  18,17,16 

5  \5  per  week      Russell  4  Lanning  ,  Units  15,  14,  11,  10, 

(non-transfer)  If termed  late  Algebra       9, 8,  7 


) 


Units  11,  10,  9  and  others 

Units  13,  12,  8,  7,  5,4,3 


non 


3 

•transfer) 


3  per  week  McCriwon 

writing  with  a  Purpose 

5  per  week  taker 
(wijth  laboratory)  The  Complete  Stylist 

3  per  week      Keedy  1  Bettinger        Units  22,  21,  20,  17 
Introductory  Algebra       (one  lesson  per  week) 

3  per  week      Wooton  &  Drooyan         Units  19,  18,  16,  15,  14 
Intermediate  Algebra       (one  lesson  per  week) 

5  per  week      Wooton  i  Drooyan         Unit's  22,  21,  20,  19,  18, 
Intermedial?  Algebra       17,  16",  15,  14 

-  7    /0  1  • 

3  per  week      Leithold  \       Jj^iti  13,  11,  10,  9,  8,  7, 
College  Algebfa    V     2,  1 

3  per  week      GallfcUnk  ]Units  13-10,  8-2 

T  ShJJfclU^eJd^ 


Think,  Read,  React,  Plan, 

Write,  Rewrite 
Ellsworth 

English  Simplified 


X 


lit  ,  Target  Courses 
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apd  on  the  perfdrmance  of  the  individual  student.     Since  the  course  wap  1$ 

I 

composition,  most  instructors  assigned  the  units  on  writing  (units  11,  10  0 
and  9)  as  course  requirements  and  supplemented  these  with  units  on  sentence 
Structure  (units  8  and  3)  for  certain  students.    The  emphasis  on  writing/.  , 
was  apparent  in  every  section,  regardless  of  the  mode  of* instruction,  by 
the  number  of  themes  which  studen^a  had  to  submit.     Often  there  we^ 
assignments  on  six  topics:    description;  natrafeion;  example  and  illustra- 
tion; comparison  and  contrast;  cjpise;,  and  effect;  ^nd  analysis.    The  primary 
textbook  for  English  111  in  ther  1975-76  academic  year  was  McCrimmoti's 
Writing  with  a  purpose  with  supplementary  use  of  Decker's  Patterns  of  * 
exposition.    .       '  .  >    '       ;     .  ^ 

In  the  "fall  quarter  of  the  1976-77  scadetikc*year  th$<^ 
to  the  use*  of  the  TICCIT  system  in  laboratory  sections  of English  111.  f 
These  sections  toet  for  five  periods  per  *50ek  but  .still  carried  ^rtiiffee  course  |, 
credits.    The  additional  periods  provided  students  weak  in  their  con^ifa  of 
English  fundamentals  with  an  opportunity  for  review.    Most  dla£&es  on  the^ 
TICCIT  system  spent  time  studying  those  units  on  grammar  and  tihci  mechanics 
>f  writing.     Instructors  generally  thought  this  t^be  ati'r^pfopriat^ ,7; 


application^of_the  TLCCIT  program  as  it  allowed  .thfem  to  focus  their  class- 
Troom  sessions  on  actual  writing.     Other  sections  of  English      1 ,  those  in  a 

lecture-discussion^format  /  followed  Baker's  The  complete  ^stvMst. 

V  '  "  *  ■  ■  -       :  ,  :. 

Enrollment  in  Targfet  Courses.     Students  had  to  retain  the  prerogative 

to  choose  their  own  courses  at  registration.     It  was  not  appropriate  jpo 

assign  students  randomly  to  the  courses  designated  as  target  courses  for 

the  demonstration.'  But  it  was  possible  to  implement  different  procedures 


JVd 


l*>3 


|rfOt  aj^££§|^  equivalent  comparison  groups  in  the  evaluation.  In  tha  first 
academt^erm  of  the  demonstration  period,  students  registering  for  algebra 
courses  at  one  site  received  thlir  section  assignments;  to  either  a  lecture 
section  or3^Pl^|rr  c^ass  ,,  on  a  random  basis.  In  other  cases  the  college's 
course  schflMtaAisted  Available  sections  without  identifying  the  instruc- 
tional condition.  Students  then  enrolled  in  a  particular  section  while 
unaware  of  its  status  as  a  lecture  discussion  or  TICCIT  class.     Later  in 

the  demonstration  period  it  became' necessary  that  course  schedules  specify 

' 

the  instructional  condition  for  all  sections  of  target  caurses.  Often 
students  unable  to  finish  course  requirements  in  one  academic  term  wished 
to  reenroll  for  the  same  course  under  the  same  co^ition  in  the  next  term. 
To  do  so  the  students  needed  to  know  whether  a  section  was  to  be  a  regular 
lecture  section  or  a  section  under  the  TICCIT  program.    These  were  the 
three  principal  methods  followed  in  placing  students  fnto  sections  of 
target  courses:  random  assignment;  blind  enrollment;  and  self-election. 

/About  one-third  of  the  students  enrolled  fot  algebra  courses  in  the 
fall  semester  at  Phoenix  College  declined  the  random  assignment  given  to 
them.  They  instead  indicated  af  strong  preference  for  a  different  ina^Tac1- 
tional  condition.  Of  those  students  who  wanted  a  lecture  section  rather 
than  a  TICCIT  class,  t^he  most  common  explanation  was  simply  a  preference 
for  the  traditional  method.     Students  also  cited  friends'  unfavorable 

reactions  to  the  TICCIT  program,  insufficient  knowledge  of 'the  prograr^alid 

»  ■  -  *•  v 

^    the  amokint  of  time  required  on  the  computer  system  as  reasons  for  declining 

an, assignment  to  the  TICCIT  condition.     Other  students  refused  an  assignment 

to*  a  Wcture  section  in  order  to*take\an  algebra  course  on  the  TICCIT 
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system.^Their  reasons  for  this  preference  were  often  tjjie  recommendations 
of  friends #  the  self-paced  instruction  available  through  the  TICCIT  system, 
and  the  lwi?e  of  an  innovative  approach  to  learning.    No  significant  differ* 
ence  fl^e  found  between  those  students  who  accepted  their  section  assignment 
and  thpse  who  expressed  a  strong  preference  for  a  different  condition  in 
terms. of  their  entrance  ability.    And  there^as  no  significant  difference 
betwfeen  these  two  groups  in  terms  of  program  outcomes. 

.    At  Alexandria  students  enrolled  in  target  courses  without  specific 
knowledge  of  the  instructional  condition.    The  course  schedule  listed 
sections  of  alglSkta  courses  as  either  "lecture"  or  "programmed"  just  as  the 

0  J  ■ 

'  sections  ha4  been  listed  prior  to  the  introduction  of  the  TICCIT  program. 

' '  •••  * ■  \         •  <  •  ■ 

Since  fthe  lise  o£  the  TICCIT  system  was  supplementary  for  the  course  in 
English  composition,  the  schedule  did  not  distinguish  sections  of  this 

course  by  condition.     As  shown  in  Appendix  K,  students  chose  to  enroll  in  a 

;       ,      ...  '  .1 

particular  section  of  a  math  course  primarily  because  they  wanted  to  take 

^the  course  undei**  the  instructional  condition  listed  for  that  s^dtion  or 
because  the  section  met  at  a  convenient  time.     In  the  fall  quarter  of  the 
1975-76  academic  year,  most  students  in  TICCIT  classes  had  been  unaware  of 
their  section's  instructional,  condition.     This  Changed  as  the  demonstration 
progressed  and  students  needed  to  know  each  section's  condition  in  order 
to  continue  studies  from  one  quarter  to  the  next*     It;  was  clear  that 
students  registered  for  their  English  classes  on  the -basis  of  the  time  of 
class  sessions.    Repeatedly  students  gave  "class  met  tt  a  convenient  time" 
as  their  basis  for  enrolling  in  a  particular  section  of  the  English  composi- 
tion course\(see  Appendix  K) .    The  students  in  TICCIT  classes  were  generally 

i 
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quite  similar  to  thos^  in  lecture-discussion  sections.    Appendix  L  contains 
profiles  of  these  two  groups  on  a  number  of  characteristics,    the  only  anomaly 
was  a  tendency  for  a  greater  percentage  of  the  students  in  TICCIT  classes 
to  be  employed  full-time.    Perttaps  students  with  j 6b*. believed  that  it  wo 
be  easier  to  arrange  a  convergent  schedule  of  classes  on  the  T^CIT  system. 

A  similar  pattern  of/choices  held  when  the  college's  course  schedule 
identified  sections  by  d»B*ition.     In  the/spring  semester  at  Phoenix 
College,  students  registered  for  certain  sections  of  algebra  courses 
according  to  the  section's  instructional  condition  and  its  hours  for  class 
sessions.     Students  registered  for  certain'  sections  ofr'the  course  in 
English  fundamentals  because  the  ^fprtnn^ir.at  citownlent.tlii^.    However,  f 
over  one-fourth  of  the  stu^e^Jn^  th*t . 

they  had  chosen  their- course  section  tfecauae-  other  s^tid^M^already 
full.'  This  happened  in;  the  £Prjta^  ye*r  ±n 

math  courses  and  again  in  the^next  £a£l, |ypmest^^ ^&isK^ourses;,  ,; 
Despite  this  apparent  dlf  f  ejia&cg  ,  in  students^  jrer* 
partictflatVsection,  there  were  jus^  small  dif^rfrjpj 
students  in  TICQIT  classes  and  *s 
differences  which  did  occur  ^t^^s 
of  the  groupp  into ' question, 

Under  all  three  procedure 
courses  there  was  a  remarkable  c|>nsia«iicy  in.  S^f 


enrolling  ii>  a 


classes.    Given  a  choice  of  conditions, stude 

ft*?*?  >  ' 

a  specific  section  by  its  manner  of  instruc 
dents* in  English  cpurses  registered  ^for(ia 


4ty  r, 
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the  convenience  of  its  sch&f&fe.    And  regardless  of  the  procedure 
in  forming  TICCIT  classes  ancl  lecture  sections,  the  comparison 
seemed  equivalent  at^^?  start  of  an  academic  term.    Further  d< 
students*  reasons  fo^^p^Jlifag  in  certain  sections  and  on  the  composition^ 
of  comparison  groups  appe^^lti  Appendix  K  and  Appendix  L.     Table  3.3  gives 
an  illustration  of  the  kind  of  information  foynd  £n  these  appendices;,  it 
presents  data  on  student  enrollment  in  target  courses  from  a  case  in  which 
students  selected  a  course  section  according  to  its  instructional  condition. 

\  i  *  .. 

3.2    Pro^ct  Administration  and  Site  Management  ^ 

What  promotes  the- transfer  of  an  innovative  curriculum  program  into 

schools?     Certainly  care  in  the  initial  selection  of  demonstration  sites 

can  facilitate  the  process  of  transfer.     Schools  able  to  accommodate  the 

program  and  known, to  be  receptive  to  its  goals  should  provide  fertile 

ground  for  a  demonstration.     But  how  do  these  schools  come  to  participate 

in  the  project  so  that^taheir  administration  and  faculty  will  both  support 


and  accept  the  new  in^jpfctional  program  as  part  of  the  school's*  curriculum? 


The  developers  6f  the, TICCIT  program^ 'took\S  Systematic  approach  to  the 
problems  of  implementation.     They  anticipated  vflh^:  issues  mi^ht  be  involved 
in  the  introduction  of  the  TICCIT  program  intp  community  colleges,  and  they 
established  formal  mechanisms  for  dealing  with  these  issues Implementation^ 
became  an  explicit  concern  in  the  administration  of  the  overall  project. 

Mb 

3. 2. a    Mechanisms  for  Implementation 

A  capsule  summary  of  the  mechanisms  established  for  facilitating 
implementation  iury.be  of  greater  import*  than  an  historical  account  oi  a 
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Reasons  for  Section  Selectiq^ 
Phoenix  College 
Math  007,  106,  KM  and  117 
Spring  Semester,  1976 


TICCIT 
(N  &  309) 

Lecture 
(Nss  553) 

N 

Percent/ 

N 

Percent 

Knew  about  Instructional  method  at  registration 

261 

84.5 

428 

80.3 

Section  was  recommended 

126 

40.8 

111' 

20.8 

*           By  classmate 

56 

18.1 

43 

8.1 

By  instructor 

46 

14.9 

30 

5.6 

*        By  counselor 

24 

7.8 

39 

'  .7.3' 

Reasons  cited  for  section  choice: 
To  take  a  course  with  TICCIT,. 
Primary  reason 
Second  most  important  factor 
To  get  a  programmed  instruction  'section 
Primary  reason 
'   Second  most  important  factor 
Preference  for  lecture/discussion  instruction 
Primary  reason 

Second  most  important  *  factor  • 
Class  met  at  a  convenient  time 
Primary  reason  w 
Second  most  important  factor 
Othefc  sections  already  full  M 
Prin^ary  il'eifeon 

Second  most  iriportant  factor 
'To  take  coursie  with  a  particular  instructor 
Primary,  reason 
Second  most  important  factor 
No  particular  reason 
Primary  reason 
Second  most  important  factor 


100 
43 


32.4 
14.6 


ft 


0.2 
0.6 


41 


4.5 
13.3 


58 
82 


10.9 
15.4 


Enrollment  in  Target  Courses 
Table  3. \ 
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Student  Demographic  Profile 

Phoenix  College 
Math  007,  106,  108,  ami  117 
Spring  Semester,  1976 


( 


Age  (years) 

College  grade  point  average 


TICCIT 
(Nat  309) 


Mean 


sd 


Lecture 
(N*533) 

Mefj^n  sd 


23.12  6.5 
2.34  .5 


24.23 
2.95 


6.7 


Total  college  credits  * 

27 

.70 

27.8 

26 

.81  23.7 

High  school  grade  po4ant  average 

2 

.75 

.6 

2 

.7^  .6- 

Sex:    .  %* 

N 

Percent 

N 

Percent 

Male 

310 

69.0 

334 

62.7 

Female 

139 

31.0 

194 

36.4 

High  school  curriculum:  * 

Genetfaf^ 

.  162 

52.4  .. 

294 

5^5.2 

Jfacatiqrtai  * 

7 

2.3 

17 

*3.2t 

College  preparatory/academic 

/ 

22.0 

23.3  1 

GED 

34 

11.0  M 

^^^^^ 

10.7 

Student  ltfad  (crettlt  hours  for  current  term): 

i 

1  J 

S.       Part-time            -  '*3*; 

92 

29. ^ 

!;   38  ;i  " 

Full-time 

201 

65.0 

324 

"      60,8  « 

Employment  (current  term) :, 
None 

Part-time 

Full-time 
Previously  enrolled  in  this  course: 

Total 
^  TICCIT 

Lecture 


81 
109 


26.2 
35.3 


128 
163 


24. 

30.6^ 


99 

32.0 

226 

42.4 

70 

22.6  ' 

89 

16.7 

50 

16.2 

26 

4.4 

9  V 

2.9 

49 

9.2 

Enrollment  in  Target  Courses 
Table  3.3(b)  , 
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project's  life.    Especially  in  the  TICCIT  project  where  the  attention 
devoted  to  implementation  was  a  formal  part  of  project  administration,* it 
is  easier  to  identify  and  relate  the  mannej  in  which  the  developers  dealt^ 
with  issues  such  as  maintaining  communication  among  participants  and 
preparing  faculty  for  different  duties  under  a  new  curriculum  program.  The 
highlights  of  the  implementation  process  appear  here  not  as, milestones  bur- 
as  specific  strategies  adopted  by  the  developers.     These  strategies  illus- 
trate both  the  range  of  problems  which  can  arise  in" a  major  demonstration 
project  and  effective  mechanisms  for  overcoming  them.    The  TICClT^^aject-, 
did- succeed  in  implementing  an  innovative  curriculum  program  in  schools;  /and 


in  completing  a  demonstration  of  that 'program's  effectiveness. 

*  College  Liaison.    A  member  of  the  developers^  staff  assumed  responsi^ 

   ~  -     .  * 

.  bility  for  liaison  with  the  community  colleges,  and  this  came  before  the 

final  serection  of  demonstration  sites.    When  two  colleges  joined  the 

.project  as^  participants,  there  was  already  a  cen^pl  point  of  contact  with 

the  developers.     Each  college  in  turn  designated  one  person  from  its 

faculty  to  -serve  as  a  local  project,  director  and  to  represent  the  college's 

interests  in  planning  the  project.     This  approach  of  appointing  one  specific 

person  as  a  channel  for  all  communications  simplified  exchanges  among 

participants.     Indeed,  the  ditfttftpinft'  .representative  made  regular  contact 

with  each  of  the  other  participants  in,  the  project  throughout  the  early 

stages  of  the  project.     Even  the  evaluator  revived  an  informal  Status 

report  through  a  semi-monthly  telephone  call  from  the  developer.  These 

^ptjjjj^st^llshed  a  regular  pattern  of  communlc  at  lot!  -among  participants. 

«   *"  ;  Appointing  a  of  implementation  for  the  developers  and  local 

project  direc^Sl$fr:  the  colleges  facilitated  planning  the  demonstration 


■4 


a?4  implementing  the  TICCIT  program  in  community  colleges,    The**  was  ready 
acdess  ^a  person  with  the  responsibility  and  authority 'to  represent  each 
participant  in  reaching  decisions,     it  also  became  a  simple  matter  to 
resolve  ambiguities  about  the  .developers'  plans  or  the  colleges'  procedures 
and  this  avoided  much  confusion  and  doubt.     Certainly  the  talents  of  the 
individuals  chosen  as  representatives  of  their  institutions  helped  to  make 
smooth  communication  among  participants  a  reality,     Qut  it  was  the  antici- 


patfon  of  a  need  fot^pegular  contacts  and  the  appointment  of  a  manager  and 
project  directors  that  put- a  mechanism  for  communication  into  place, 

^  Coordination  Committee,  Agreements  reached  with  the  colleges  as  part 
of  their  formal  commitment  to  the  project  stipulated  that  there  would  be  a 
committee  formed  to  coordinate  all  project  activities.    The  committee  ^ 

m ' 

included  the  person*  designated  ,as  project  directors  for  the  developers, 

J** 
for  the  evaluator,  as  well  as  th6  presidents  of  the 

colleges  and  a  senior  vice-ptesiden£  fgt£  vthe>  developer .     Its  membership 
reflected  the  committee's  level  of  responsibility  in  reviewing  the  progress 
of  the  project  and  taking  \mmediate  action  in  order  to  resolve  any  problems, 
Its  purview  toSH^Ln  the  project's  adherence  to  milestones,  each  partici- 
pant's fulfillment  of  obligations,  and  the  financial  resources  available  to 

S 

the  project.    The  ^ommittee  met  twice  in  each  yjJx  of  the  project. 

Complications  usuall^  occur  in  a  large  d Jvelopmentfal  project  under  a 
tight  schedule,  and  the  TICCIT  project  Vas  no  exception.     Frank  discussions 
pf  problems  and  priorities  within  the  coordinating  committee  helped  to 
identify  those  issuer  which  required  imfifediate  at  tendon.    The  committee's 
members  had  sufficient  authority  to  take  action  and  make^commitments  for 
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their  institutions.    It  was  possible  to  resolve  conflicts  before  they 

seriously  threatened  the  project's  success  in  mounting  a  demonstration. 

This  senior  level  committee  was  probably  essential  to  the  coordination  of 

the  project  among  participants.  (  • 

Implementation  Plan.     Before  community  colj^gefrentered  into  ah 
agreement  to  join  in  the  projects  demonstration  sit.s;  the  developers  of 
the  TICCIT  program  began  to  prepare  an  Implementation  plan.    The  purpose  of 
the  document  was  to  provide  participants  with  specific  detail|concerning 
the  preparation,  installation- and  operation  of  TICCIT  systems  at  community 
colleges.    Drafting  the  Implementation  plan  over  one  year  in  advance  of 
the  use  of  the  TlCCI^&taW in  schools  at  least  made  participants 
sensitive  to  the  scope  of  work  entailed  in  the  demonstration.    And  both  the 
developers  and  the  colleges  JfBgan  to  consider  ways  to  accomplish  specific  ; 
&sks  wel| -In  advance  of  ^eir^hedule  f  or  ^completion.    Table  3. A  lists 
the  sections  of  the  implementation  plan  and  the  participant  responsible  for 

ft"  each  section.  )  > 

Although  the  plans  as  set  forth  in  the  Implementation  document  did  not 
always  provide  the  best -match  for  later  conditions,  these  plans  did  represent 
a  sound  basis  for  entering  into  the  demonstration.     The  Implementation  plan  , 
had  passed  the  approval  of  each  participant  and  so  provided  written  docu- 
mentation on  procedures  and  operations.     If  the  developers  and  the  colleges 
disagreed  on  a  crucial  point  of  the  Implementation  plan,  a  final  decision 
was  left  to  the  coordinating  committee.     Here  was  a  reference  and  guide  *or 
the  implementation  pK^ess  and  the  conduct  of  the  demonstration:  . 

That  muchof  the  implementation  plarf  later  becam^bsolete  or  inappro- 
<riate  could  be  expected  given  the 'evolving  nature  of  an%mbitious  prefect.  ?  At 


157  - 


Section 


Topic 


Responsible  Party 


A 
B 

C 
D 
E 


G 
H 
I 
J 
K 
„  L 
M 

!  N 
.  0 
P 


Management  and  Organization 
Schedule  of  Activities 
Support  Requirements 
Facility  Preparation 
Training 


Coordination  (Courseware  Development 
and  Implementation) 


Curriculum  IntegrajgLon  ^  ^ 

Installation  and  Checkout 
Operations 

Contingencies  *  \ 

System  Maintenance 
Formative  Evaluation 
Summative  Evaluation 
Visitor  Control 
Supporting  Documentation 
Definitions 


Developer 


Developer  v* 

Community  Colleges 

Community  Colleges 

Curriculum  Developer  and 
-    Community  Colleges 

Curriculum  Developer 


Community  Colleges 
Develop^r^^  ^ 
Community  Colleges 
Community  Colleges 
Developer 

Curriculum  Developer 
Evaluator 

Community  Colleges 


Table  3.4 
Contents  of  Implementation  Plan 
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the  outset  of  its  preparation  the  developers  thought  it  would  be  necessary 
to  update  and  revise  the  plan.  That  the  bulk^f  the  document  actually  did 
reflect  the  st«ps  taken  in  implementing  the"  tfetf  T  program  is  a  tribute  * 


5^'  \%o  it*  usefulness. 


fin-unity  Collie  Authors.    Four  faculty  members  from  the  community 
colleges  relocated  to  the  site  of  courseware  production  M**  period  of  ^ 
*  '     months.     There  was  one  representative  from  the^ mathem^tW  department  and 
another  from  the  English  department  of  each  college.     They  had  the  famili- 
arity with  community  college  students  and  the  knowledge  of  the  colleges- 
curriculum  structure  which  the  developers  themselves  lacked.    The  developers 
understood  this  weakness  and  had  invited  faculty  from  the- community,  colleges 
to  spend  the  second  half  of  the  1972-73  academic  year  with  the  courseware 
production  teams.     These  community  college  authors  played  an  important  role 
-       in  revising  the  initial  content,  specif ications  for  the  TICCIT  curriculum  so 
that  they  would  closely  conform  to  the  content  of  the  target  courses. 

Beside,  contributing  to  the  content  specif ications  ofS:he  courseware, 
the  community  college  authors  made  the  TICCIT  program  a  produj*  of  the 
dellopers  and  the  colleges.     There  could  have  been  great eV  faculty  resist- 
anofto  implementing.  «.  curriculum  program  from  an  outside  source  had'  not 
their  colleagues  he Htffcft' shape  and->ef ine  the  program.     And  the  involve- 
i     Hit.  of  community  college  authors  set  a  precedent  for^ater  efforts.  As 
the  project  came  nearer  to  the' demonstration  phase'with  some  tasks  still 
incomplete,  members  of  the  mathematics  departments  of  the 

V         community  colleges  devoted  substantial  time  and  effort  to  readying  the*f 
curriculum  for  students.     Enlisting  com^rtity,  col^e  authors  was  a  . 


r 
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Mjpr  step  toward  gaining  faculty  acceptance  o#  t^^tifclt  program  and 
reaching  a  set  of  content  specif icatlona  appropriate  for  the  colleges. 

Faculty  Training  Program*    A  summer  program  was  offered  to  faculty 
members  in  order  t6  prepare  them  for  the  use  of  the  TICCIT  system.  The 
training  program  took  place  in  the  summer  of  1974  just  aq  the  colleges 
started  experimental  classes  based  on  the  TICCIT  curriculum.     Bight  faculty 
members,  five  math  instructors  and  three  English  instructors,  attended 
an  intensive  series  of  -  presentations  at  one  of  the  community  colleges  .  ' 
Three  math  instructors  attended  a  parallel. program  given  at  another  of  the 
community  colleges.     The  developers  of  the  courseware ^conducted  these 
sessions  over  a  period  of  almost  threl  weeks  at  each  college;  they  also 
provided  a  manual  with  sections  on  an  introduction  to  the  TICCIT  program, 
administrative  procedures,  handling  student  problems  and  'authoring. 

"The  sessions  did  familiarize  faculty  with  the  TICCIT  system,  particu- 
larly  its  design  and  its  capabilities.     But  there  was  no  opportunity  to 
prepare  instructors  for  their  responsibilities  in  TICCIT  classes.  Neither 
the  facilities  nor  the  experience  existed  for  such  preparation.  The 
computer  system  was  not  yet  fiilty  Operational  and  much  of  the  courseware 
remained  unavailable  dt  the  community  colleges.    Moreover,  ther£  had  not 
been  a  course  under  the  TICCIT  program  gjven  anywhere  which  coi^d  serve  a 
model..  Faculty  had  to  learn  about  their  new  role  as  instructors  in  TICCIT 
erases  as  they  began  to  use  the  TICCIT  program  in  the  pilot  year,  the 
1974-75  academic  year,  of  the  project.    Had  there  been  both  an  initial 
orientation  session  to  acquaint  faculty  fully  with  the  TICCIT  system  and 
a  later  program  to  prepare  instructors  for  their  responsibilities  in  TICCIT 


classes,  with  this  progtam  offered  after  the  experience  of  a  modest  field 

trial,  it  might  have  been  e^ier  for  the  developers  to  define  the, t#acher' a 

f  I  < , 

role  under  the  TICCI1  program  and  simpler  for  the  faculty  to  learn  about 

the  computer  system/a|d  the  course  materials  as  they  worked  at  a  terminal. 

Stud  en  t»  Handbook.    The  student  handbo/Sk'  was  intended  to  serve^  as  a 

reference  and  «uide  for  students  enrolled  in  a  mathematics  or  English 

course  under  the  TICCIT  program.    It  contained  the  3ame  statements  of 

objectives  *nd  subject  matter  rules  that  appeared  on ^e^J^  syste».' 

But  the  handbook  had  them  in  a  written,  bound  form  familiar  tt>  Students. 

There  were  few  examples  and  no  practice  problems  in  the  handbook  since  its 

function  was  as  a  supplement  to  the  courseware  available  on  the  computer 

system  rather  thkn  as  a  textbook  for  3tudent  learning  away  from  the  system. 

Few  students  actually  purcha3ed  the  handbook.     If  a  student  was  already 

on  the  TICCIT  system,  the  handbook  was  redundant.     If  a  student  was  at  home; 

the  handbook  could  not  serve  as  a  sourcebook  for  studying  examples  or  for 

working  problems.    To  replace. the  handbook  the  colleges  developed  a  set  of  * 

basic  instructions  for  student  use  of  the  TICCIT  system.     These  instructions, 

distributed  to  all  students  in  a'course  under  the  TICCIT  program,  covered 

the  mechanics  of  accessing  courseware,  course  requirements,  and  standards 

for  grading.  "They  provided  mechanical  and  administrative  information  which 

complemented  the  instructional  material  available  on  the  TICCIT  system.  < 

3.2.b  Staffing 

The  amount  of  staff  necessary  for  the  TICCIT  program  is  a  critical 

,  v  *  V  s 

issue  since  a  major  goal  of  the  project  was  to  demonstrate  a  cost-effective  , 

"  application  of  computer-assisted  instruction.    1^ order  to  be  cost-ef fectiv* 
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the  TICCIT  program  would  need  to  operate  under  a  much  higher  student-fatuity 
.  ratio  than  regular  classes  or  result  in  substantial  gains  in  outcomes  such  * 
as  course  completion  rates  and  student  achievement.    Seldom  does  a  school 
define  its  student-teacher  ratio  in  such  a  way  as  to . include  support  staff 
such  as  counselor*  and  librarians.    This  makes  it  difficult  to  compare  the 
staff  complement  behind  the  TICCIT  program  with  an  equivalent  for  the 
traditional  lecture-discussion  mode  of  instruction.    Here  we  cannot  even 
offer  a  definition  of  the  staff  required  for  the  TICCIT  program.  What: 
follows  is  simply  a  descrip:y.on  of  the  staffing  adopted  by  the  two  community 
colleges  in  the  demonstration. 

Staff  with,  responsbilities  related  to  the  TICCIT  program  fell  into, 
three  categories:    administrative,  instructional  and  technical  staff.  The 
project  director  of  each  college  was  the  administrative  staff.    This  person 
reported  to  the  college's  president  and  collaborated  closely  with  the 
developer  and  evaluator  on  all  project  activities.    The  responsibility  for 
implementing  the  TICCIT  program  within  the  college  rested  with  the  project 
director,  along  with  the  budget  and  technical  details  of  the  program,  'once 
the  program  had  been  implemented  and  the  demonstration  completed/the 
position  of  the  college's  project  director  became  obsolete.  Presumedly, 
the  TICCIT  program  would  then  be  well  integrated  with  the  college's  opera- 
tions and  accepted  as  part  of  the  curriculum  structure,  and  there  would  no 
longer  be  need  of  a  college  project  director. 

Faculty  members  and  proctors  provided  instructional  support  for 

students  on  the  TICCIT  system.     Faculty  maintained  their  affiliation  with 

/' 

an  academic  department  and  taught  TICCIT  classes  as  party' of  their  course 


4 
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load.    In  a  sense  there  Here  no  TICCIT  teachers  since  no  instructor  taught 
only  classes' on  £he  TICCIT  system.  4  Instead,  members  of  the  mathematics  and 
English  departments  accepted'  assignments  to  TICCit  classes  and  received 
credit  toward  their  full-time  load  of  coijrses.    The  totTal  number  of  load 
hours  granted  to  instructors  with  assignments  to  TlCCIT  classes  was  the 
equivalent  of  about  three  full-time  faculty  positions  **t  each  college. 
This  figure  reflects  assignments  to  both  mathematics  and  English  courses 
during  the  demonstration  phase  of  the  project. 

A  section  of  an  English  course  given  on  the  TICCIT  system. carried  the. 
same  number  of  load  hours  and  contained  roughly  tTie  dame  number  <of  students 
as  any  other  section.  .  Thus,  five  sections  of  '?a  three-credit  English  course 

*.  J 

under  the  TICCIT  program  amounted  to  the  equivalent  of  one  full-time 
faculty  position.    The  usual  ratios  of  etuderits  to  teachers  and  of  faculty 

load  hours  to  classroom  contact  hours  held  for  those  sections  of  English  „• 

c  9  ui    "  ^  *  V 

courses  offered  on  the  TICCIT  system.,  /  ^ 

A  math  instructor  received  only  three-fourths  or  four-fifths,  depending 
on  the  college,  of  a  faculty  load  hour  for  each  contact  hour  spent  with 
students  on  the  TICCIT  system.    While  a  lecture  section  of  Algebra  I  would 
yield  five  load  hours,  the  same*  course  taught  under  the  TICCIT  program 
would  carry  four  load  hours.    This  policy  had  been  adopted  by  the  colleges 
because  math  classes  on  the  TICCIT  system  presumedly  required  no  outside 
preparation  of  lectures  or  grading  of  homework  assignments.     Classes  of 
Algebra  I  under- a  programmed  instruction  format  also  carried  less  than  one 
load  hour  pe^  contact  hour.    A  full-time  facult^r  position  in  mathematics 
usually  entailed  teaching  five  sections  of  a  three-credit  course;  the^ 


equivalent  of  a  full-time  Instructor. could  cover  six  sections  under,  tl 
TICCIT  progra^.'    '*;•/,.  '   «'  '  '  ,.  I  \ 


The  two  .communjity' colleges  differed  in,  their  approach  to  staffing 
sections{  of  matheinaticjB  courses  on  the  TICCIT  system.    At  Alexandria 4 in- 

•  /  -v  -;  *  .  ;  •     •  •    ,    '  > 

structors  assto^  Responsibility  for  specific  TIC€lT  classes  just  as' they  ■., 

did  for  lecturei  sections.    However,  classes  on  the  TICCIT  system  pf^n* 

included  some  students  registered  for  Algebra  I  'and  others  registered  for' 

Algebra  II.  .  The  TICCIT  prograil  enabled  the  college  to  combine  students 

enrolled  for  different  courses  under  ttfe  sam>^ihs^^uctor ,  tHus  forniing  one 

full  section  from,  multiple' courses  rather  than  canceling  two  or  more  partial 

sections,    i At  Phoenix  the'  m^hematlcs  faculty/ achieved' £ven  greater  flexi- 

i  ,  <  f 

bility  through  the  TICCIT  program .  ,  Siudents^arra^e^/their  own  hours  on 

the  TICCIT  system  rather  than  registering  tor  partdyfcular  sections,  and  in- 

structors  were  present  during  the  specified  hours  of  system  operation.  Since 

both  students  and  instructors  chose  heure  on  the> TICCIT -system,  there  was  no 

intact  section  of  a  group  of  students,  wttjf.an  instructors,    Instead  students 

relied  he4vi4y  on  the  TICCIT  program  for  their  learning .  The\tudent-teacher 

ratio  reiterates  the^  Importance  of  the  TICCIT  program as  the  pfcimary  resource 

for  instruction.    Although  TICCIT  classes  of  Alexandria's  Algebra  I  and  / 

Algebra  II  conformed  to  the  usual  size  of  lecture  sections^ 0-2 5  students), 

there  were  sometimes  70  students  working  on  the  TICCIT  system  for  Phoenix' 

mathematics  courses  and  only  one  member  /of  the  mathematics  department 

3  -    "     < '  /         '  ** 

presenter  support.    An  increased  student-teacher  ratio  and  (a  reduced 

ratio  of  faculty  load  hours'  to  student  contact  hours  meant  that  instructors 

) 

could  serve  a  larger  number  of  students  under  the  TICCIT  program. 


.  -  •     .  ■  .       '•  .: .... 
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While  instructors  responded  to  students'  questions  about  the\  subject 

matter,  proctors  showed  students  how  to  use  the  TICCIT  system.  Frequently 
proctors  circulated*  among  'students  and  sought  out  ajudents  in  need  of 
aasistance.    They  served  as  a  link  to  instructors,  referring  all  content" 
questions  to  a  faculty  member  and  resolving  mechanical  or  procedural 
difficulties  by  themselves.    The  proctor  acted  in  a  role  very  much  like 
that >of/a  teacher's  assistant.  'Actually* the  two  proctojjs  hired  at  Alexan- 
dria'had  qualifications  well,  above  the  minimum  requirements  for  the  position: 
one  had  three  years  of  experience  in  teaching  at  the  secondary-school  level 
and  another  was  a  graduate  student  in  a  doctoral  program  in  educational 
""technology.    The  project  director  at  Phoenix* adhered  closely  to  the  initial 
specifications  for  the  position  as  undergraduate  students  fulfilled  the  ^ 
proWor's  functions  for  the  TICCIT  program.    These  undergraduate  students 
worked^n  a  part-time  basis ,  with  perhaps  a  dozen  students  ^assuming 
the  equivalent  of  four  full-time, positions.    A  description  of  both  proctor 
and  instructor's  activities  appears  **t*r  in  this  repprt  (see  Chapter  8: 
■  •  Faculty  Acceptance  and  Teacher  Role) i 

Two  other  staff  members  prx>vJ^ed  the  'support  necessary  f or ^he^computer 
system  and  its  peripheral  devices.-  They  acted  as  computer  operators  and  / 
•     systems  engineers  in  maintaining  the  TICCIT  system  and  in  keeping  the 
system  operating  "in  a  reliable/manner.     Seldom  was  there1  any  lengthy 
interrupt Igfrlxi  system  service  during  the  demonstration  phase  of  the  TICCIT 
.project.  .Most  technical  difficulties  had  been  resolved  in  the  initial  year 
"*    of  piloArials  for  the  TICCIT  program,  and  the  computer  technicians  . 
'•'at  the  colleges  had  learned  procedures  for  resolving  problms^quickly./v 
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All  together  there ^were  at  least  eight. college  positions  directly 
.related. to  the  TICCIT  program:    one  project  director;  three  faculty  members; 
two  proctors;  and  two  technicians.  'With  this  staff  complement  the  TICCIT 
program  could  not  be  cost-effective.    The  slight  reduction  in  instructor 

w  costs  due  to  the  larger  number  of  ^students  or  classroom  hours  assigned  to  1 
mathematics  faculty  did  not  compensate  for  thfe  increase  in  other  staff 
costfc.    By^  the  pattern  ,for  staffing  adopted  by  the  two  community  colleges 
in  the  demonstration,  need  not  dictate  the  procedures  followed  elsewhefre. 
These  simply  represent  two  cases  of  wfcat  was  done  in  order  to  support  the 
TICCIT  program,  and  neither  case  really  is  faithful  to  the  developers' 

"projected  staffing  pattern..  However,  there  should  be  Attention  given  to 
the  results this  evaluat^n, -particularly  those  invplving  £ourse.  completion 
rates, 'before  advocating  further  reductions  in  instructional  st*f£  as  a 
natural  consequence .of  computer  applications  to  teaching, 

r'1  • 

\  - 

3.3    Courseware  \  .  f 

'  f  !  )  i 
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The  courseware  employed  on  the  TICCIT  system  contributed  as  much  to 
t^e  differences  observed  between  TICCIT  classes  and  lecture-discussion 
sections  as  any  ot\^x  <|ompojient  of  the  program.     Students  enrolled  for 
courses  under  the  T^CIT  program  received  instruction  both  innovative  in 
'its  design  and  un^Jxal  in  its  manner  of  delivery.    Whether  the  computer  as 
a  delivery  mechanism  or  learner  control  as  a  design  strategy  led  to  the  . 
differences  reported  in  this  evaluation  is  a  moot  question.  ^The  TICCIT 
program  had  been  conclived  as  a  systematic  approach  to  computer-assisted 
instruction  with  eac/h  of  its  components  dependent  on  another.  Learner 
control  would  be  impractical  without  computer  support,  and  a  computer 
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ayete.  developed  for  a  particular^  of  curriculum  materiale  would  be  of 
1**U  value  without  thoae  materU^.    The  TICCIT  program  should  be  viewed 
aa  a  whole,  and  the  cpuraewar^epreaenta  not  a  separate  component*  but 
rather  a  highly  visible  aspectT of  system  capabilities. 

This,  sectioL  is  a  ^discussion  of  th<*  TICCIT  courseware:  *  its  design 
and  its  production.    More  complete  accounts  of  the  design  and  production  of 
courseware  for  Jhe  TICCIT  project  already  exist  (Bunder son,  1972,  1975). 
The  description^ven  here  just  toshes  upon  the  processes  involved  in 
courseware  develop^  in  order  to  convey  a  sens^  of  £ose  efforts  and^n 
appreciation  of  the, TICCIT  curricu|um.    ,  ^  S*. 

3. 3. a    Instructional  Design  ,  *  ('  \ 

A  basic  assumption  underlying  the  deJign/of  the  TICCIT  courseware 
was  that  the  instructional  design  followed  in  developing  curricula  it  inde- 
pendent of  the  subject  matter/  If  Jhe  manner  which  a  topic  is  taught  is 
really  independent  of  the  topic  itself,  then  .the  instructional  d*±gn  can 
be  the  same  regardless  of  "topics  or  subjects."  Itje  developers  of  the  TICCIT 
program  applied  the  same  instructional  desi«f^il^leasons  in  mathematica 
and  English.  This  eliminated,  the  need  to  specify^  instructional  strategy 
for  *each  separate  lesson  and  aimpli'fied  the  process  offcourseware  production. 

The  instructional  strategy  adopted  for  the  TICCIT  courseware  depended, 
(n  m'aps  and  learner  control;    Maps  lent  structure  to  the  courseWre^y 
organizing  the  content  at  variojs levels  of  detail  and  by.  charting  the 
aubject  matter  to  be  learned,    Learner  control  gave  students  co^and  over 
their  learning  oy  allowing Wto  choose, -^th>n  constraints,  tonijcsVr 
atudy  and  sequences  for  instruction.    Throt/gh  ma^s  And  learner  control  the 


TIdCiT  [jrogram  laid  the  responsibility  for  learning  ,on  the  individual 
student.    The  program  "was  a  passive,  sophisticaterf^esource.  for  learning 
rather  than  an  active  intervention  for  teaching. 

Ce^tbooks  offer  a  familiar  parallel  fp*s*inder standing  the  Structure  of 
TICCITjf  courseware Just  as  a  textbook  has^chapte^rsr^t^ei^ons  within  " 
chapters,  the  T1CC1T  courseware  has  units  and  Wol  within  units,  Maps 
in  the  courseware  fulfill  much  the  same  fanWon,  ^  tables  of  contents  in 
textbooks:  "  they  provide  a  dire^fcory  o,f  potent  coverage.'  A  table  of 
counts,  however,  simply  lis  J^'ery 'leve.l/of  a  -textbook's  material  in  an 
online  format.     The  maps  on.  the'  TICCIT  s^em  present  a  visual  picture  v 
Qf,thi  content  coverage  a%  a^ngl^&vel  oKdekil.    For  example,  a  course 
^ma*  would*show  all  of  the!  units  requi^  for  ^ourse^fl^  Figures  1.3 
an*  1.4  show  all  of  the  un^ts*  in  th6  mathematics  apo'English  courseware. 
#tidJ>eyond  ttfie  course  map  there  would  be  other  maps  for  each  unit  and  then 
each  lesson,  lading  tfce  ^udent  through  successive  levels  of  detail  in  the 
-courseware.  * 

the^e  maps  guided  and , directed  student  learning.    By  depicting  the 
formal  relationships  inh«#enj:  to  the  subject'mat  tjeh  and  employing"  the  color' 
capabilities  of  the  TlCClT  system,  maps  told  students  what  material  they 
had  already  mastered  and  ^ere  they  should  go  next.    A  student  would  first 
encounter  a  course  map  displaying  the  units  of  that  course,  such  as  the 
course  map  for  College  Algebra  given  in  Figure'3.'l.    Units  toward  the 
bottom  of  the  display  serves  pveve/qut^es  Cb^hose  which  appeared  above 


them.    .Thus  the  unit  on  relations  Vunit  11)  was  a  prerequisite  to  the 
on. functions  and  conic  sections  (Units  9  and  10),  and  the  unit  on  functions 
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13.  Matrices 

1.  Probability  . 

2.  Sequences  and  Permutations 


Figure  3.1 
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gma  in  turn  «  prerequisite  to  the  unit  on  polynomial  functions  (Unit  8).  A 
unit  box  colored  green  meant  that  a  student  had  demonstrated  mastery  of 
that  topic  and  could  proceed  on  to.  another  topic;  yellow  indicated  a  need 
for  further  work  and  ,red  was  a  sign  of  weakness  in  student  performance. 
These  cues  given  by  placement  and  color  on  a  display  and  color  held  across 
course,  unit  and  lesson  maps.    And  the  sequence  of  maps  shown  in  Figure  3.1 
represents  the  progression  of  three  maps  at  the^course,  unit  and  lesson 
level!  necessary  for  accessing  a  particular  lesson  in  a  course,  in  this 
case  Unit  10  Lesson  4  in  College  Algebra. 

In  their  form  apd  purpose  maps  on  the  TICCIT  system  resembled  learning 
hierarchies  (Gagne,  1970).     Like  learning  hierarchies  they  imposed 
structure  on  the  subject  matter  by  suggesting  prerequisite  relations  among 
topics.    This  constrained  student  choices  of  what  material  they  could  study 
since  a  red  entry  on  a  map  prevented  access  to  any  component  of  the  map 
directly  above  it.    Similariy,  a  yellow  box  limited  access  to  adjacent 
topics  while  a  green  box  permitted  the  continuation  of  studies  along  the 
hierarchy.     It  was  not  until  after  a  lesson  map  had  appeared  that  instruc- 
tion took  place.    Lesson  maps  contained  the  finest  level  of  detail  on  the 
TICCIT  system*,  segments. 

The  structure  of  the  courseware  gradually  led  to  finer  pieces  of 
content:     course,  unit,  lesson  and  segment.     Each  successive  level  could  be 
defined  in  terms  of  the  next*    But  segments  could  not  be  reduced  further." 
They  were''  the  building  blocks  of  the  courseware.    At  the  segment  level  the 
strategy  of  learner  control  truly  began.  ~" 

Basically,  each  segment  consisted  of  a  rule  accompanied  by  examples 
and  practice  problems  involving  the  application  of  that  rule.  Rules 
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presented  students  with  a  definition  of  a  concept  or  a  statement  of  a 
principle,  as  illustrated  by" the  rulesgiven  in  Figure  3.2.    Along  with 
every  rule  there  was  a  collection  of  instances  of  its  use.    An  instance 
together  with  a  solution  served  as  an  example  of  the  rule's  application;  an, 
-incomplete  instance  called  for  an  answer  from  the  student  and  provided 
practice  (see  Figure  3.3  for  illustrations  of  each  type  of  instance). 
Both  examples  and  practice  problems  could  be  drawn  from  the  same  file  of 
instances.    This  basic  rule-example-p«ctice  paradigm  certainly  appears 
elsewhere.     It  had  been  iopted  nearly  a  decade  prior  to  the  TICCIT  project 
as  an  approach  to  programmed  instruction  (Evans,  Homme  &  Glaser.,  1962). 
What  distinguished  the  TICCIT  program  from  its  predecessors  was  its  combina- 
tion of  learning  hierarchies  and  learner  control. 

Students  on  the  TICCIT  system  had  control  over  the  rules,  examples  and 
practice  problems;  presented  to  them.    The  student  rather  than  the  computer 
chose  the  sequence,  difficulty  level  and  frequency  of  rule  and  instance 
^displays.     Commands  corresponding  to  the  three  elements  of  a  courseware 
segment  were  included  oh  the  keyboard  for  TICCIT  terminals  (see  Figure 

0 

1.2).     By  entering  a  segment  from  a  lesson  map  and  pressing  RULE,  EXAMPLE 
or  PRACTICE.'a  student  could  exercise  control  over  an  instructional 
sequence.     Further,  the  TICCIT  system  permitted  the  student  to  determine 
the  number  of  examples  and  practice  problems  through  repeated  key  presses  ^ 
as  well  as  their  difficulty  level  through  other  commands,  namely  the  EASY 
and  HARD  keys.     For  example,  a  student  might  first  request  a  statement  ofr 
the  rule  (RULE),  view  several  examples  of  a  use  (EXAMPLE),  proceed  to  try  a 
number  of  practice  problems  (PRACTICE)  and  progress  to  more  difficult 
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A  circle  with 
center  (h,k)  and 
radius  r  is  the^ 
set  of  alA  points 
whose  distance  to 
(h,k)  is  rA 


RULE 
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Definition  of  a  Concept 


To  correct  a  MISLEADING  MISPLACED 
MODIFIER: 


1.    Move  it  closer  to  the  element 
it  modifies 


2.%  Rewrite  the  sentence  if  the 
modifier  is  still  awkward. 
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Statement  of  a  Principle 


Figure  3.2 
Rule  Displays 
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The  radius  of  a  circle  Q  is:  3 
Its  centar  ia:  d/2»"2) 

The  equation  of  the  circle  ia: 

<x-V2)2+(y-2,)2  -  3 


£XAMP  1'  HARD  PAGE  1/2 


Example 


Paul  looked  at  the  pie  that  was 
sitting  on  the  table  hungrily. 

The  sentence  above  may  or  may  not 
have  modification  problems,  Mark 
each  vord  in  any  DANGLING  or 
MISPLACED  MODIFIERS.     If  there 
are  none,  Just  ENTER, 


PRACT  5  EASY  PAGE  l/l  PART  1/2 


Practice  Problem 


Figure  3.3 
Instance  Displays 
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problems  (HARD) •   -Alternatively,  a  student- confident  in  his  or  her  knowl- 
-  edge  -of  the  -topic  could  start  a  segmen^  by  answering  practice*  problems  or 
a  student  unsure  about  a  concept  might  want  to  review  a  large  number  of 
examples  at  different  levels  of  difficulty  befote  attempting  any  problems, 
(Descriptions  of  students'  use  of  the* learner  control  option*  available  on 
the  TICCIT  program  can  be  found  in  Chapter  7:     Student  Activities,)  When 
a  student  had  demonstrated  mastery  by  responding  cbrrectly  to  a  sufficient 
number  of  practice  problems  with  the  correct  answers,  the  student's  lesson 
map  became  green  for  that  particular  segment. 

Courseware  maps  took  students  through  the  structure  of  the  TICCIT 

s- 

curriculum  and  left  them  at  segments.    At  this  level  the  learner  control 
of  the  TICCIT  system  came  into  play.    The  options  available  to  each  student 
affected  the  sequence,  difficulty  level  and  frequency  of  rule  and  instance 
displays.    The  TICCIT  system  provided  students  with  the  opportunity  and 
responsibility  to  take  command  over  the  instruction  presented  to  them. 
Additional  features  on  the  TICCIT  system  helped  students  in  managing  their 
learning.    There  was  a  component  of  learner  control  which  gave  further 
assistance  upon  student  request  (HELP).     Such  assistance  could  result  in  a 
simpler  explanation  of  a  rule,  a  gradual  and  detailed  presentation  of  an 
example,  or  an  answer  to  a  practice  problem.     If  a  student  was  uncertain 
about  what  to  do  next,  the  system  could  suggest  an  appropriate  choice 
dependent  on  that  student's  previous  performance  (ADVICE).    And  the  system 
had  videotape,  color,  graphic  and  audio  capabilities^for  strengthening 
its  presentations. 
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3.3.b    Courseware  Production  ' 

«  The  uniform  nature  of  the  Instructional  design  adopted  for  the 
TICCIT  program  made  it*  p£%sibie  'for  the  courseware  developers  to  establish 
a  production  model  for  the  jcurriculum.    Regardless  of  subject-matter,  all 
^courseware  adhered  to  the  same' form  of  map  structures  and  Included  the  same 
features  at  the  segment  level.    This  consistency  In  design  encouraged  the 
assembly  of  courseware  production  teams,  with  each  member  of  the  team 
assigned  specific  tasks.    The  teams  brought  together  persons  with  different 
fields  of  expertise  so  that  members  could  contribute  their  own  special  i 
skills  to  resolving  problems  in  curriculum  development.     Such  an  approach 
was  a  marked  departure  from  usual  methods  of  developing  curriculum  materials. 
It  was  an  explicit  attempt  to  establish  a  model  for  the  efficient  production 
of  high  quality  curriculum  materials  for  computer-assisted  instruction. 

A  fresh  approach  to  curriculum  development  was  demanded  by  the  ambi- 
tious scope  of  the  TICCIT  project.    Within  two  years  courseware  sufficient 
for  supporting  at  least  four  full  courses  in  mathematics  and  English  had  to 
be  completed  and  set  in  place  at  community  colleges.    And  numerous  questions 
affecting  the  development  of  those  materials  had  to  await  progress  in  devel- 
oping  the  computer  system  and  in  arriving  at  content  specifications  in  line 
with  the  colleges'  curriculum.    There  had  to  be  a  courseware  production 
model  capable  of  rapid  curriculum  development.    The  courseware  developers 

hoped  to  achieve  this  efficiency  with  production  teams  and  standard  forms 

) 

for  all  materials. 

Each  production  team  was  to  include  representatives  of  six  specialized 
skills.    An  overview  of  these  six  types  of  team  members  and  their  respective 
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responsibilities  appears  in  Fi^re  3.4.     Since  they  had  actually  specified 
the  instructional  design  for./the  TICCIT  program,  the  instructional  psycho- 
logists naturally  assumed  responsibility  for  insuring  that  all  courseware 
adhered  to  the  structure  of  maps  and  to  the  strategy  of  learner  control . 
Persons  fulfilling  this  role  brought  exceptional  strengths  in  educational 
psychology  and  computer  applications  to  the  production  teams. 

Much  of  the  real  burden  of  curriculum  development  fell  upon  the 
subject  matter  experts.     They  supplied  the  content  specifications  for  the 
courseware  and  developed  the  maps  of  content  coverage,     in  developing 
courseware  maps  the  subject  matter  experts  began  at  the  level  of  the  full 
curriculum  (see  Figures  1.3  and  1.4)  and  proceeded  down  to  the  level  of 
segments  where  specific  rule  statements  and  instances  of  their  application 
appeared.    These  members  of  the  production  team  typically  had  advanced 
degrees  in  either  mathematics  or  English  and  had  taught  college  courses  in 
their  particular  field.    The  community  college  authors,  those  instructors 
released  for  six  months  to  join  the  production  teams,  functioned  as  subject 
matter  experts. 

Instructional  design  technicians  provided  a  critical  link  between 
curriculum  development  and  courseware  production.    They  received  contetit 
specifications,  in  the  form  of  courseware  maps  with  some  exemplary  rule 
statements,  from  the  subject  matter  experts  and  then  expanded  upon  these 
rough  drafts  so  that  other  members  of  the  production  team  could  construct 
the  files  of  rules,  examples  and  practice  problems.     This  required  defining 
the  critical  attributes  of  each  rule  and  developing  specific  guidelines  for 
the  construction  of  different  instances  of  its  use.     The  definition  of  each 
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Team  Member 
Instructional  Psychologist 


Subject  Matter  Expert 


Instructional  Design  Technician 


Major  R 
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Authoring  Assistant 


Packaging  Specialist 


Empirical  Design  Technician 


Providing  guidance  in  the  production 
of  courseware  consistent  with  the 
instructional  design  of  the 
TICCIT  program. 

Structuring  the  curriculum  in 
terms  of  courseware  maps  and 
drafting  statements  of  the  rules 
and  concepts  to  be  learned. 

Completing  specifications  for 
rules,  examples  and  practice 
problems  iti  accordance  with 
the  options  .built  into  the 
strategy  of  learner  control. 

Generating  files  of  rules  and 
instances  to  meet  the  courseware 
specifications • 

^ Preparing  curriculum  materials 
for  display  by  computer. 

Validating  the  instructional 
quality  of  the  courseware. 


Courseware  Production  Team 
Figure  3.4 
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rule's  attributes  had  to  be  fittf*  enough  to  suggest  what  courseware  should 

be  made  available  in  response  to  jthe  various  options  of  TICCIT's  learner 

'»  j 

control *(e.g,,  HELP,  EASY,  HARD).    Graduate  students  in  instructional 
psychology  served  as  instructional  design  technicians  in  developing 
these  detailed  courseware  specifications. 

Next  in  the  flow  of  courseware  production  came  the  authoring  assistants 
and  the  packaging  specialists .    Authoring  assistants,  usually  chosen  on  the 
basis  of  an  undergraduate  degree  In  the  subject  matter  and  relevant  experi- 
ence, received  the  courseware  specifications  and  generated  the  necessary 
rules  and  instances.    Packaging  specialists,  typically  undergraduate 
students  employed  on  a  part-time  basis,  polished  the  filial  product  and 
generally  made  the  courseware  read£  for  computer  display.    The  supervisor 
of  these  specialists  had  to  understand  the  display  capabilities  of  the 
TICCIT  system,  such  as  its  color  capacity,  graphics  support  and  audio 
messages,  and  decide  when  it  was  appropriate  to  use  them.     Finally,  the 
empirical  design  technician  was  to  validate  the  instructional  quality  of 
all  materials. 

From  a  simplified  description  of  the  production  team  it  may  seem  as 
though  the  process  of  curriculum  development  was  itself  smooth  and  orderly. 
Seldom  did  the  reality  of  curriculum  development  closely  resemble  the  "ideal" 
production  model.    That  original  model  of  a  factory  for  courseware  produc-  ' 
tion  (Low,  Graves,  Stilson  &  Jacobsen,  1972)  had  gone  so  far  as  to  specify 
the  piecemeal  assembly  of  a  curriculum  at  20  work  stations.    There  were  to  , 
be  certain  members  of  the  production  team  responsible  for  completing  a 
given  task  at  each  work  station.    The  detail  and  specificity  involved  in 
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courseware  production  perhaps  facilitated. the  process  of  curriculum  de- 
velopment but  also  almost  certainly  contributed  to  some  of  the  problems 

ear*  ,  '  \ 

encountered  by  the  developersv  »    *    .  7 

Without  closure  on  the  instructional  design  of  /the  T1CCIT  program  or 
the  capabilities  of  the  T1CCIT  computer  system,  interruption  and  delay  were 
inevitable.    The  most  persistent  problem  in  the  early  history  of  the  T1CC1T 
project  was  the  concomitant  development  of. the  computer  syst.em  and  the 
curriculum  materials.    Often  the  engineers  did  not  know  what  specifications 
their  computer  system  had  to  meet  whil^the  courseware  developers  were 
uncertain  about  what  could  be  expected  .of  the  computer  system.     And  the 
physical  separation  of  the  system  engineers  and  courseware  developers  only 
aggravated  their  respective  uncertainty.    The  courseware  developers  had  to 
.commit  themselves  to  an  instructional  design  in  order  to  define  the  computer' 
support  which  would  be  required  by  the  curriculum.    There  was  no  opportunity 
to  benefit  from  trial  and  error  In  defining  the  initial  T1CCIT  system. 
Quite  obviously  a  prototype  computer  system  for  exploring  the  ramifications 
of  different  alternatives  in  instructional  design  would  have  been  desirable. 

At  times  the  specialization  inherent  to  the  approach  to'  curriculum 
development  as  a  production  process  impeded  progress.    Therejft|  insufficient 
interaction  among  members  of  the  production  teams.     Each  spelKlist  tended 
to  contribute  just  one  type  of  expertise  without  appreciating  how  that 
expertise  might  best  blend  into  the  production  process.    Yet  a  knowledge 
of  the  entire  process  was  essential  if  a  subject  matter  expert,  for  example, 
was  to  develop  content  specifications  that  could  later  take  advantage  of 
the  display  capabilities  of  the  computer  system,  since  colored  graphics 
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could  be ^powerful  cues  in  learning  or  fanciful^  abstractors  of  student 
attention.    Deciding  how  different,  specialties  might  complement  rather  than 
detract^fxom  one  another,  and ^recognizing  what  constrains  "had  to  be  met  at 
various  stages  in  the  process  of  curriculum  development,  required  close 
collaboration  among  members  of  the  production  team's. 

x      Therfe  were  also  frequent  changes  in  the  roles  and  responsibilities  of 
te#m  members  in  response  to  schedule  demands  and  sta^f  attrition.  As 
deadlines  approached  for  the  completion  and  release  of  courseware,  subject 
matter  experts  became  team  leaders  responsible  for  implementing  ^the it  own 
production  models  and  for  producing  particular  lesson*,     dc^ubject  matter 
expert  found  it  most  expedient  to  act  as  an  individual  author  since 
he  had  a  strong  command  of  the  entire  production  process.    Another  subject 
matter  expert  chose  to  stay  with  the  original  production  model  and  rely 
heavily  oh  other  members  of  his  production  team.  \  And  a  third  production 
team  had  a  succession  of  subject  matter  experts,  each  with  a  somewhat 
diff4rent  view  of  an  appropriate  structure  for  the  courpeWare.    The  original 
production  model  seemed  too  cumbersome  for  easy  adaptation  to  tight  dead-'  j 

lines  or  staff  wo^k  styles.  \ 

>  *  ■ 

Validation  of  the  courseware  took  place  at  the  community  colleges  as 

,    "     .       v  -  :*\  ".«"♦ 

faculty  reviewed  the  courseware  and  students  took  courses  under;  the  TlC^rf 
program  on  an  experimental  basis.     Formative' evaluation  become  largely  a 
matter  of  identifying  errors  and  recommending  changes  to  strengthen  the 
curriculum.     The  courseware  developers  made  the  necessary  revisions  and  ) 
considered  further  improvements  if  their  resources  would  permit  the  changes. 
In  the  place  o£,a  formal  evaluation  prior  to  the  demonstration  phase  of  the 
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project  there  j^As  extensive  gj^view  and  revision.    Resourcea^ad  tfb  be 


{  devoted  to  the  completion  of  all  courseware  rather  than  £gp  Refinement. 
Thus  , there  was  scant  opportunity  to  validate  m^tery  f|f&8  embedded  &  the 


lesson  level  or  to  prove  the  eff icady  of  t^^^purseWjtf^maps  a&  learning 
hierarchies.     Instead  a  puLoKJL  judgments  oj^^^  ^^rmance#  ct^titutee  ^ 
mastery  had  to  be  accepted  and  ixi±t±Bl^j^^^ox^Wj^^  the  logical  struc- 
ture  of  the  subject  matter  had  tc 

Despite  these  prbblemft^  completed  a  curriculum 


equivalent  to  five  fujl  cour^e*^^  instruction  possible  in4 

those  mathematics  and  Bngl^Lflh  courSfei  with  the  highest  studfent  enrollments. 
And  it  gave  students  cqntrol  djV^^ieir  own  Instruction  along  with  resppnsi- 
bility  for  their  ovfh  learning ♦^Th"e  final  product  embodied  the  expertise  of 


•Mr- 


divers^  disciplines  and  repiresfchted  the  culmination  of  an  innovative  and 


ambitious  prpjeqt.* 


,y  •       »  k 


3 • 4    Summary  ^ 

The  developers  of  the  TI CC IT  program  approached  the, problems  of 
implementing  computer-aBSisted  'in&trjiction  in^schools  in  a  systematic 
manner.    Each  step  in  the  process  oi, introducing  thejTICCIT  program  into 
the  school  curriculum  ha^  been  carefully  planned.    There  was  a  sound 
'  rationale  f  or  choosing  commifn|*ty  col^egfes  >s  the  level  of  education  most 
conducive  to  a  market  success  for  comp\$ter-ass is ted  instruction.    And  there 

ejjjpUcit  criteria  for  selecting particular  community  colleges  as 
demonstration  sites  for  (hjq  .jgtoject.    :To  maintain  the  commitment  of  these 
sites  and  to  prepare  theii^f  oryfhe  a<jbpti<in'  of  the  TICCIT  program,  the 

'"  ;  ;'  <    '  *     »"      '       iM!    '•'  ' 

developer^  established  *p#clfcic  procedure?/   These  procedures  involved  the 

.  *  ■    •  ;••    -i  *%•  '  :.  . ■  % 
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community  colleges  in  the  administration  of  the  project  on  a  regular, 

»      active  basis.     Similarly,  the  design  *and  production  of  the  TICCIT  course- 

.    .  ■  >' 

X  .  ware  reflected  a  systems  approach  to  curriculum  development.    All  materials 
made  available  under  the  TICCIT  program  followed  the  same  instructional 

'"a      ,  • 

design.     Such  consistency  in  design  enabled  the  developers  to  specify  a 
^single  production  model  for  the  entire  curriculum.     In  all  respects, 

including. curriculum  development,  the  TICCIT  program  was  a  systematic 

.      ■  ft 

application  of  computer-assisted  instruction. 

Enrollment  in  community  colleges  had  tripled  in  the  decade  prior  to 
the  stWt  of  the  TIGCIT  project.    Projections  indicated  that  it  would 
double  within  another  decade.    Unlike  other  sectors  of  education,  there  was 
rapid  growth  in  student  enrollment  at  community  colleges.     Furthermore,  a 
high  proportion^  the  total  enrollment  occurred  in  introductory  mathematics 
and  English  courses.     By ^kcentrating  on  just  a  few  select  courses,  the  . 


luppHmt 


TICCIT  program  could  sypj>Hnft  as  much  as  one-fifth  of  a  college's  traditional 
curriculum.    This  4*ould  simply  be  equivalent  to  full-time  students  taking 
one  mathematics  or  English  course  under  the  TICCIT  program  in  each  academic 
term  of  their  studies.'  And  the  policy  of  open  admissions  so  common  at 
community  colleges  had  created  'a  persistent  search  for  better  ways  to  teach 
students  basic  skills  in  mathematics  and  English.     Community  colleges, 
tpey  represented  the  high-volume  market  necessary,  for  commercial  success 
with  computer-assisted  instruction. 

The  particular  community  colleges  selected  as  demonstration  sites  had 
to  meet  certain  criteria.  Foremost  among  these  criteria  for  selection  was 
geographic  proximity  to  the  developers.     The  installation  of  a  new  computer 
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system  and  the  introduction  of  a  new  curriculu^program  were  apt  to  require 
heavy  external  support  for  the  community  colleges.    Another  important 
factor  taken  into  consideration  was  the  college's  capacity  to  accommodate 
the  project.    There  had  to  be  sufficient  physical  space  for  the  computer 
sys'tem  and  student  carrels,  competent  staff  available  to  support  the 
program's  usfe ,  and  a>  high  enough  enrollment  in  the  target  courses  to 
justify  a  need  for  128  terminals-     Several  colleges  otherwise  qualified  to 
become  participants  in  the  project  had  student  populations  unlike  the 
national,  norms  in  their  characteristics.     The  developers  of  the  TICCIT 
program  sought  and  found  demonstration  sites  that  would  be  representative 
of  community  colleges  in  general. 

Formal  agreements  reached  with  the  demonstration  sites  stipulated  that 
there  would  be  several  steps  taken  to  insure  the  smooth  and  orderly  transi- 
tion of  the  TICCIT  program  into  the  college  curriculum.    Notable  among 
these  were:     the  appointment  of  local  project  directors  at  the  colleges  and 
a  person  responsible  for  college  liaison  for  the  developers;  the  establish- 
ment of  a  committee  with  senior  representatives  of  each  participant  to 
oversee  the  project;  the  development  of  an  iinplementation^jjpri  with  specific 
details  on  the  preparation,  installation  and  operation  of  TICCIT  systems  at 
community  colleges;  and  the  participation  of  community  college  instructors 
in  developing  content  specifications  for  the  TICCIT  courseware.     Each  of 
these  steps  helped  to  make  the  community  colleges  active  participants  in 
the  project  with  a  definite  role  in  project  administration  and  planning. 
Later  the  colleges  devoted  significant  amounts  of  their  own  resources 
toward  completion  of  the  TICCIT  program. 
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In.  order  to  develop  a  curriculum  equivalent  to  five  full  community 

/" 

college  courses,  the  developers  of  the  TICCIT  courseware  adopted  a  systematic  ^ 

( 

%  \ 

approach  to  the  design  and  production  of  curriculum  materials.     Each  \ 
component  of  the  TICCIT  curriculum,  regardless  of  content  coverage,  adhered 
to  the  same  basic  instructional  design.    That  design  depended  heavily  on 
maps,  similar  to  learning  hierarchies,  to  lend  structure  to  the  subject 
matter.     A  series  of  three  maps  depicting  progressively  finer  levels  of 
detail  in  the  subject  matter  graphically  charted  the  content  coverage  of 
TICCIT  courseware.    Learner  control  became  available  at  the  finest  level  of 
detail,  the  segment.     Each  segment  contained  a  rule  along  with  examples  and 
practice  problems  illustrating  the  application  of  that  rule.  Through 
options  built  into  ftie  TICCIT  system  a  student  could  exercise  control  over 
the  sequence,  difficulty  level  and  frequency  of  rule  and  instance  displays. 

The  uniform  nature  of  this  instructional  design  across  the  TICCIT 
courseware  invited  the  application  of  a  single  production  model  for  the 
entire  curriculum.    The  courseware  developers  assembled  production  teams 
inciud^g  members  with  different  specialty  skills,  and  they  defined  a 

production  process  specifying  the  flow  of  the  curriculum  along  work  stations. / 

< 

This  approach  to  curriculum  development  was  meant  to  facilitate  the  rapid 
production  of  high  quality  courseware.     It  did  encourage  contributions  from 
persons  wi^h  different  fields  of,  expertise,  and  it  did  result  in  the 
com|i|£tion^f  a  curriculum  comparable  in  coverage  to   five  full  courses. 
But  ^here  were  numerous  changes  to  the  original  production  model  along  the 
way  to  these  accomplishments.     Some  became  necessary  because  courseware 
production  took  place  concomitantly  with  development  of  the  computer  ' 
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system.    Others  occurred  because  the  production  model  was  too  rigid  for 
adapting  easily  to  schedule  demands  and  work  styles. 

This  chapter  has  concentrated  on  the  proceeds  involved, in  Implement- 
ing the  T1CC1T  program  at  community  colleges  and  in  developing  its  curric- 
ulum*   Also  presented  were  descriptions  of  the  student  populations  at  the 
demonstration  sites,  the  target  courses  for  this  evaluation,  the  procedures 
for  student  placement  in  comparison  groups,  and  the  colleges'  staff  for  the 
TICCIT  program.     The  chapters  which  follow  report  the  results  attained  by 
the  TICCIT  program  in  terms  of  its  impact  on  student  performance. 
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Chapter  4 
Course  Completion  Rates 

Although  student  achievement  is  generAly  accepted  as  the  single,  most 
important  measure  of  program  success,  there  is  an  even  more  direct  measure 
of  educational  impact.     It  is  simply  whether  or  not  a  student  completes  the 
work  required  under  a  program.     In  a  way  this  is  a  very  rough  indicator  of 
achievement  since  any  passing  grade  signifies  completion  and  all  else  becomes 
noncompletion.     Yet  it  is  an  indicator  with  direct  consequences  for  students 
and  schools.     A  student  who  fails  to  complete  a  course  may  not  advance  to 
higher  level  courses  in  the  same  subject  or  perhaps  not  fulfill  requirements 
for  promotion  or  graduation.     Repetition  of  courses  or  failure  to  satisfy 
school  standards  in  turn  affect  a  school's  productivity:     what  proportion  of 
students  starting  a  program  of  studies  actually  completes  it. 

Such  completion  rates  seem  eapecially  important  for  the  community  college. 
Here  is  an  institution  which  must;  cope  with  a  heterogeneous  student  body, 
often  with  deficiencies  in  basic  skills,  and  prepare  these  students  for  an 
occupation  or  further  academic  work.     The  TICCIT  program  addressed  precisely 
those  skills  which  represent  a  significant  component  of  the  community  college 
curriculum  and  seem  always  to  call  for  improved  teaching  methods.     The  colleges 
that  participated  In  the  TICCIT  project  did  so  at  least  in  part  to  find  improved 
teaching  methods  for  their  courses  In  algebra  and  composition.     And  the  colleges 
definitely  sought  a  currlcular  program  that  would  enable  a  larger  percentage  of 
students  to  complete  courses  in  algebra  and  writing.     Indeed  completion  rates 
above  those  associated  with  lecture  classes  waqp  one  of  the  stated  goals  of  the 
TICCIT  program. 
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Besides  representing  program  productivity  and  participants'  concerns, 


completion  rates  can, affect  the  composition  of  our  comparison  groups.  Some 


students  certainly  will  not  finish  their  Course  and  so  be  absent  fi  m  post- 
testing.     As  long  as  the  characteristics  of  those  present  for  posttesting 
remain  independent  of  their  instructional  condition,  the  comparison  of  student 
achievement  under  one  program  with  that  under  another  should  be  fair  and 
accurate.     Even  when  programs  lead  to  significantly  different  completion  rates, 
achievement  analyses  can  be  free  of  bias.     It  is  when  course  completion  depends 
on  a  student  trait,  such  as  entrance  ability,  under  one  condition  but  not 
another  that  possible  contamination  of  other  data  occurs.     Then  achievement 
analyses  would  be  subject  to  the  programs^  differential  effects  on  completion, 
and  completion  rates  would  be  the  only  unbiased  measure  of  program  impact. 

For  these  reasons  our  first  chapter  on  the  outcomes  of  the  TICCIT  projeck 
concerns  course  completion  rates.     It  begins  with  a  section  about  the  meaning 
of  completion.     Completion  is  not   the  simple  complement  of  attrition  nor  is  it 
an  entirely  unambiguous  measure.     So  we  need  to  explain  what  we  mean  by  course 
completion  rates.     The  chapter  continues  with  a  description  and  analysis 
of   results.     Given  quite  different  results  dependent  on  the  course  conditions 
(i.e.,  classroom  instruction  or  TICCIT  program)    it  is  appropriate  to  take  » 
completion  a  step  further  and   look  at  enrollment  trends  in  the  next  academic 
term.     This  -^h&pter  (loses  with  a  discussion  of   the  factors   that  contributed 
to  our  findings  and  the   implications  of   these  results. 

4 . 1  The  Nature  of  Completion 

Admittedly  completion  might    be  del ined   in  several  diflerent   ways.  No 
one  definition  is  likely  to  fit  all  circumstances  or  suit  all  purposes.  With 
that   in  *ind   let  us  state  our  preference:     a  course  completion  rate  is  the 
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ratio  of  students  who  receive  credit  for  a  course  to  all  those  who  initially 
enroll  for  t^e  course.    On  a  final  grade  sheet  it  is  the  number  of  passing 
grades  in  relation  to  total  class  size.     For  example  if  a  class  has  32  students 
listed  on  the  official  college  rolls  and  24  of  them  receive  A,B,C  or  D  grades, 
the  completion  rate  would  be  .75  or  75%  of  initial  enrollment.    It  is  a  seem- 
ingly simple  concept.    But  completion  viewed  in  this  manner  aid  attrition  taken 
as  students  who  drop  out  of  a  course  hardly  account  for  all  Students. 

Especially  at  colleges  with  a  policy  of  open  enrollment  and  a  commitment  to 
service  for  the  local  community  there  is  a  sizeable  percentage  of  students  some- 
where between  completion  and  attrition.    These  students  attend  classes  regularly 
and  yet  fall  short  of  fulfilling  requirements  for  course  credit.    In  one  sense 
such  students  have  completed  the  course  since  they  were  in  attendance  at  both 
the  start  and  end  of  an  academic  term.    Obviously  they  did  not  drop  out  of  the 
course  and  so  would  not  be  part  of  attrition.    But  completion  should  imply  some- 
thing beyond  mere  attendance:     it  also  suggests  that  a  student  has  satisfied 
all  course  requirements.     It  is,  as  we  said,  a  measure  of  productivity.  Neither 
the  studeri/  who  withdrew  from  a  course  or  the  one  who  failed  to  meet  course 
standards  for  credit  would  qualify  as  a  completion. 

This  may  appear  to  be  a  particularly  harsh  definition  for  programs  which 
purport  to  allow  for  individual  differences  among  students.     Such  programs  • 
frequently  let  a  student  proceed  through  a  course  at  his  own  pace.    A  school 
might  then  assign  a  nonpunitive  grade  to  students  who  fail  to  finish  all 
of  the  required  course  work  within  an  academic  term.    Typically  this  grade 
carries  no  penalty  in  terms  of  grade  point  average  or  class  standing  but  makes 
,  it  necessary  for  students  to  enroll  in  the  course  again  and  complete  all 
,  3f°rk  in  order  to  receive  credit.    A  reenroll  grade  is  a  step  toward  recognition 
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of  differences  among  students  in  their  learning  rates.    It  might  then  lie 
argued  that  it  is  inappropriate  to  evaluate  completion  rates  under  a  self- 
paced  system  of  instruction  like  the  TICCIT  program  within  the  fixed  time 
constraints  of  an  academic  term.    Certainly  the  argument  is  a  legitimate 
one  if  the  proportion  of  students  who  eventually  achieve  mastery  matches  J 
or  exceeds  that  for  completion  in  a  lecture-discussion  condition. 

All  of  our  tables  with  summary  figures  on  completion  therefore  present 
alternative  interpretations  for  calculating  rates.    One  alternative  gives 
the  benefit  of  doubt  about  a  student's  eventual  status  to  the  TICCIT  program. 
It  counts  students  still  attending  classes  at  the  end  of  a  term  as  completions. 
The  analyses  for  completion  as  a  program  outcome,  however,  use  the  more  restric- 
tive definition  of  fulfilling  all  requirements  for  course  credit  within  an 
academic  term.    But  these  analyse*  do  include  a  variable  that  would  indicate 
the  success  of  students  reenrolling  in  a  course  apart  from  those  just  starting. 
We  favor  a  restrictivevinterpretation  of  completion  as  it  reflects  productivity 
within  fixed  time  constraints  and  is  borne  out  by  data  across  terms. 

4.2  College  Grades  and  Rosters 

The  source  of  our  data  about  students  completing  a  course  was  the  college 
roster.    An  emphasis  other  than  the  receipt  of  course  credit  might  lead  to 
a  different  approach  to  documenting  completion.    If  our  primary  interest 
were  attendance,  it  would  be  more  appropriate  to  keep  daily  attendance  records. 
These  would  show  attrition  as  well  as  the  regularity  of  attendance  clearly. 
Should  we  accept  some  minimal  level  of  student  achievement  as  evidence  of 
completion,  then  posttesting  would  be  sufficient.    But  the  official  college 
rosters  seem  to  incorporate  these  features  in  varidus  course  grades  and  the 

final  roster  with  grades  represents  the  definitive  statement  on  student  status* 
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Probably  every  type  of  administrative  record  has  inaccuracies  that  occur 
somewhere  along  the  line  in  its  construction.    As  a  general  rule  the  care  taken 
in  the  preparation  of  final  grades  avoids  etfror  on  college  rosters.    What  did 
occur  in  our  data  were  inconsistencies  in  section  assignments  and  grade  require- 
ments.   Sometimes  an  instructor  will  ask  a  colleague  to  admit  a  student  registered 
for  one  class  into  another.    Such  an  occurrence  is  infrequent  but  it  would  seem 
from  Table  4.1  that  this  courtesy  reached  epidemic  proportions  at  Alexandria 
in  introductory  algebra.    Actually  it  was  quite  easy  for  an  instructor  to 
recommend  that  a  student  encountering  some  difficulty  try  the  course  under 

a 

ij  ''' 

an  independent  study  format.    These  exceptions  to  the  instructional  condition 
that  held  for  the  rest  of  the  students  in  a  section  came  to  our  attenticm 
through  other  sources  of  evaluation  data  (e.g.,  online  records,  student  inter- 
views).   And  although  21  students  listed  in  lecture  sections  pursued  their 
course  work  under  programmed  conditions,  individual  cases  like  these  stood  with 
the  condition  corrected  in  our  analyses  for  completion  as  a  program  outcome. 
None  of  the  math  courses  at  Phoenix  or  the  English  courses  at  eithet  demonstra- 
tion site  had  inconsistencies  in  section  assignments  even  close  in  total  to  ^ 
those  of  the  introductory  algebra  course  at  Alexandria. 

Another  kind  of  inconsistency  within  sections  involved  class  size. 
It  too  occurred  primarily  at  one  college  in  one  subject.    Often  the^c^ass 
rosters  released  early  in  the  academic  term  listed  a  larger  number  of  students 
than  the  later  rosters  with  final  grades*    Students  who  never  came  to  a  class 
or  dropped  shortly  after  the  term  began  were  deleted  from  the  official  rosters. 
Thus  later  rosters  gave  accurate  class  sizes,  and  obscured  possible  differences 
in  attrition  early  in  a  term. 
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Math  31:   Algebra  I  at. Alexandria 
Academic  Year  1975-76 


Instructional  Condition 
for  Course  Sections  f 

Instructional  Condition  for  Individual  Students 
(Exceptions  to  Class  Condition) 

TICCIT       Programmed  Lecture 

TICCIT 

Fall  Quart$|^:  Programmed 

Lecture 

6  2 
*               2               _  0 
0  15 

TICCIT 

Winter  Quarter  Programmed 

Lecture 

10  1 

o       "■    _  tr 
\            1            1  - 

TICCIT 

Spring  Quarter  Programmed 

Lecture 

4  0 
3  0 
1                  5  i 

TICCIT 

Totals  Programmed 

• 

Lecture 

20                                 ?  r 

5  *  0 
2  21 

Exceptions  to  Class  Conditions 

Table  4.1  % 
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Across  sections  there  were  differences  in  grading  standards.  One  instructor 
might  assign  A  grades  to  a  much  larger  percentage  of  students  than  another* 
instructor  would.    Whether  a  grade  was  an  A,B,C  or  D  had  no  effect  on  our 
dichotomized  variable  for  completion.    This  equivalence  of  all  passing  grades 
cancelled  out  most,  if  not  all,  variations  among  instructors  in  grading  practices. 
Even  if  an  instructor  chose  to  give  a  student  who  fell  short  of  fulfilling 
course  requirements  a  grade  usually  reserved  for  withdrawal  instead  of  one 
better  suited  to  reenroll  status,  there  would  still  be  no  effect  in  terms  of 
our  preferred  definition  for  completion.    Any  grade  other  than  those  that 
carried  course  credit  simply  indicated  a  zero  value  for  our  completion  variable. 

.  Simplifying  grades  to  only  two  values,  omits  some  information.  There 
were  really  five  distinct  and  different  grades  assigned  by  instructors. 
Firstly,  the  grades  that  went  to  student^  who  fulfilled  course  requirements 
and,'  therefore,  received  credit  were  A,B,C  and  D.     Secondly,  the  grade  associ- 
ated with  unacceptable  work  and"  failure  was,  naturally,  F.     In  the  developmental 
math  sequence  at  Alexandtia  these  two  extremes  were  S  for  satisfactory  performance 
and  U  for  unsatisfactory.    A  grade  of  W  for  withdrawals  was  a  third  classifica- 
tion shared  by  both  colleges.     It  usually  meant  that  a  student  had  changed 
^ections,  withdrawn  from  the  course,  or  been  dropped  by  the  instructor  due  to 
excessive  absences.  Occasionally  a  student  wotffd  miss  a  final  exam  or  neglect 
ju^st  one  aspect  of  a  course's  requirements,  or  come  very  close  to  completing 
all  course  work  under  independent  study.     Such  students  received  a  grade  of 
incomplete  (I  at  Alexandria  and  X  at  Phoenix)  that  forced  them  to  fulfill  their 
outstanding  course  obligations  in  order  to  receive  credit.    This  fourth  type  of 
grade  was  given  only  when  instructors    xpected  students  to  complete  the  course 
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within  a  matter  of  a  few  weeks  at  most.    It  reverted  to  a  withdrawal  on  the 
student' 8  record  if  he  did  not  complete  his  outstanding  obligations  by  a 
deadline  prescribed  "by  college  policy.    In  cases  where  students  had  made 
progress  in  the  course  but  not  enough  to  warrant  either  a  grade  with  credit  or 
an  incomplete;  instructors  gave  a  grade  requiring  students  to  enroll  in  the 
course  again  in  order  to  receive  credit.    These  reenroll  grades  (R  at 
Alexandria  and  Z  at  Phoenix)  were  common  among  students  in  the  independent  study 
programs  already  at  the  colleges.    Alexandria  offered  introductory  algebra  in  a 
programmed  i^^:  ruction  format  and  Phoenix  had  an  audio-tutorial  program  for  its 
sequence  of  algebra  courses. 

Of  course,  college  rosters  with  final  course  grades  were  not  the  sole 
source  of  our  data  about  completion.    They  provided  our  measure  of  the  program 
outcome  taken  as  having  a  value  of  one  for  students  whd  received  course  credit 
and  zero  otherwise.    For  this  to  be  a  fair  representation  of  an  effect  dependent 
on  the  treatment  we  also  needed  comparison  groups  formed  on  a  random  basis  or 
at  least  measures  suitable  for  identifying  similar  students  in  the  groups. 
Students  at  Phoenix  were  randomly  assigned  to  conditions,  subject  to  their 
consent,  in  the  fall  semester  of  1975.    In  other  terms  procedures  were  at  best 
quasi-random  and  so  background  data  such  as  grade  point  average  and  sex  were 
collected  in  addition  to  routine  pretests.    These  independent  variables  served 

as  rough  benchmarks  in  checking  group  similarity  and  as  covariates  in  estimat- 

\  .  _  ■ 

ing  treatment  effects. 

I 
\ 

4.3  Presentation  of  Results 

As  part  of  the  evaluation  there  were  routine  summaries  of  grade  distribu- 
tions and  course  completion  rates.    These  were  prepared  from  the  college 
rosters  issued  immediately  after  the  close  of  an  academic  term.    The  rosters 

224 


had  final  grades  for  that  academic  term  and  therefore  reflect  completion  within 
that  fixed  time  constraint.    No  attempt  was  made  to  update  these  rosters  later 
since  those  grades  subject  to  change,  namely  incompletes,  represented  only  about 
five  percent  of  the  total  distribution.    And  our  emphasis  was  on  completion  as 
a  reflection  on  productivity.     Still  it  is  worthwhile  to  consider  the  possible 
changes  if  all  students  assigned  a  grade  of  incomplete  subsequently  met 
their  outstanding  requirements  and  received  course  credit.    One  of  our  alterna- 
tive interpretation* 'for  completion  goes  even  further  and  credits  grades 
of  incomplete  and  reenroll  as  completions  from  the  standpoint  of  attendance. 

Each  summary  of  course  grades  also  presents  three  completion  rates. 
The  first  corresponds  to  our  preferred  definition  of  completion  as  the  propor- 
tion of  grades  with  credit  to  total  enrollment.    The  appropriate  calculation  is 
then  simply  the  number  of  A,B,C  and  D  grades  divided  by  the  total  number  of 
Students  listed  on  the  official  class  roster.    This  interpretation  makes  no 
distinction  between  the  total  number  of  students  registered  fox  a  course  and 
the  number  who  actually  attend  classes  and  put  forth  an  effort  toward  learning. 
A  second  rate  therefore  incorporates  an  approximate  correction  for  early 
withdrawal  and  nonattendance.    Our  version  of  a  corrected  completion  rate 
again  has  the  number  of  grades  with  credit  as  its  numerator  but  the  denominator 
is  reduced  by  the  number  of  withdrawals.    The  expected  result  of  this  adjustment 
is  always  a  higher  estimate.    The  most  optimistic  and  least  stringent  view  of 
completion  is  our  third  rate,     it  is  the  ratio  of  grades  with  credit  plus 
grades  of  '^complete  or  reenroll  to  total  class  size.     In  one  sense  this 
represents  an  attendance  rate  as.  its  complement  is  largely  the  rate  of  withdrawal. 
In  another  it  represents  a  maximum  completion  rate  independent  of  time  if  We 
assume;  that' all  students  with  incomplete  and  re  enroll  grades  eventually  earn 
course  credit • 


These  three  completion  rates  as  well 'as  grade  distributions  by  section 
and  totals  by  condition  appear  in  Appendix  M.    There  is  a  table  for  each 
course  in  which  the  TICCIT  program  constituted  a  curricular  alternative.  These 
figures  cover  the  implementation  and  demonstration  periods  of  the  project •  It 
might  be  noted  that,  in  this  appendix  and  elsewhere,  our  references  to  sections 
through  a  convention  of  arbitrary  numbers  remain  consistent.    Thus,  result© 
in  terms  of  completion,  achievement  and  attitudes  can  be  cross-ref erenc^d Tror 
a  particular  section. 

To  elaborate  now  on  results  for  individual  sections  would  only  further 

<  ti 

delay  presenting  data  on  program  outcomes.     It  is  probably  sufficient  to  v 
cite  a  few  general  trends.    Especially  for  math  courses  t^heire  was  a  high  degree 
of  consistency  among  sections  under  the  same  condition.     Completion  rates 
for  classes  on  the  TICCIT  system  seem  relatively  stable  across  section?, 

-p. 

courses  and  colleges  with  slight  increases  across  terms.    However,  vther$  was  a 
difference  in  section  rates  dependent  on  subject  matter.     Course^  in  t^ri 
usually  led  to  a  somewhat  higher  percentage  of  studgpts  with  credit  than  m 
courses.     Indeed  the  proportion  of  students  with  credit  more  than  doubled 
between  math  and  English  classes  on  the  TICCIT  system.  « Generally  there  was 
greater  consistency  across  subjects  under  the  lecture-discussion  condition. 
Within  a  course  certainly  there  was  much  less  variation  among  sections  under 
the  same  condition  than  between  conditions.    These  trends  appear  more  clearly 
in  the  summary  tables  that  follow. 

4. 3. a  Math  Courses 

Tables  4.2  and  4.3  present  the  grade  distributions  and  course 

completion  rates  for  math  courses  at  Phoenix  and  Alexandria.     Because  students 

were  randomly  assigned  to  comparison  groups  id  the  fall  semester  and  nearly 
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*''Tern  and 
Course 

Semester: 

Math  007 


Math  106 


Math  108 


Com^rison 
Group 


Total 

TICCIT 

Lecture 

Toial 

TICCIT 

Lecture 

Total 

TICCIT 

Lecture 


Soring  Semester: 
Math  007 


Mach*l'06, 


Math  108 


Math  117 


Total 
TICCIT 
Lecture 

Total 

TICCIT 

Lecture 

Total 

TICCIT 

Lecture 

Total  * 
TICCIT 
*.  Lecture 


Summer  Semester 
'    Hath  106  N  \  , 

■  V.  '■ 

Math  117 


Total 
; TICCIT 
;  tecture 

/ 

Total 
TICCIT 
Lecture 


Number  of 


Count  Completion  Rates 

Phoanix 
Math  007,  106,  108  and  117 
Academic  Yaar  1973-76 


4*it 


Credit 
(A,B,C,D) 


Grade  Distribution 

Failure      Incomplete  -Re-enroll 

(?) 


(X) 


(2) 


Withdraw 
(H) 


Completion 
Rates 


tudents 

N 

X 

N 

e/ 
~" 

N 

X 

n  i 

* 

1 

TT 

TTT 

lit 

572 

196 

7 

1 

8 

1 

202 

35 

159 

28 

.34 

.47 

.71- 

215 

27 

life 

0 

2 

1 

136 

63 

50 

23 

.13 

.16 

.77 

357 

169 

47^| 

2 

6 

2 

66 

18  • 

109 

31 

.47 

.68 

.68 

i  r\  *i 

30/ 

107 

35  •> 

2 

1 

2 

1 

104 

■34 

92 

30 

.50 

.69 

144 

34 

24 

0 

0 

1 

1 

80 

56 

29 

20 

24 

163 

73 

45 

2 

1 

1 

1  . 

24 

15 

73 

39 

.45 

.73 

,6p 

160 

52 

32 

0 

0 

2 

1 

■  62 

39 

44 

27 

.32 

.45 

.72 

76 

10 

13 

0 

0 

2 

3 

52 

68 

12 

16 

11 

at 

84 

42 

50 

0 

0 

0 

a 

,  10 

12 

32 

.50 

.81 

.62 

•  • 
484" 

181 

37 

6 

1 

25 

5 

134 

28 

138 

29  ,< 

.37 

.52 

.70 

195 

31 

16 

2 

1 

23 

12 

82 

42 

J f 

90 

1  £ 

00 

OA 

289 

150 

52 

4 

1 

2 

1 

1ft 

AO 

01 

aO 

•  Ji 

70 

•71 

298 

101 

34 

2 

1 

19 

6 

84 

28 

91 

37 

.34 

♦49 

.68 

135 

15 

11 

0 

0 

19 

14 

61 

45 

40 

30 

.11 

.16 

'.70 

163 

86 

53 

2 

1 

0 

0 

23 

14 

51 

44 

.53 

♦77 

.67 

201 

62 

31 

1 

0 

14 

7 

60 

30 

63 

31 

.23 

.34 

.60 

85 

11 

13 

1 

1 

13 

15 

45 

T  c 
i  J 

1  ft 
XQ 

•  1J 

.16 

•  ol 

116  ' 

51 

44 

0 

0 

1 

1 

..  15 

13 

48 

41  ' 

.44 

.75 

.58 
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115 

52 

0 

0 

12 

5 

33 

16 

62 

28 

.52 

.72 

.72 

61 

14 

23 

0 

0 

11  , 

18 

16 

26 

20 

32 

.23 

.34 

.67 

161 

101 

63 

0 

0 

,1 

1 

17 

11 

42 

26 

.63 

.85 

.74 

131 

49  .37 

0 

0 

19 

15 

39 

30 

24 

18 

.37 

.46 

.82 

50 

8 

16 

0 

0 

19 

38 

12 

24 

11 

22  ' 

.16 

.21 

.78 

81 

41 

51 

0 

0 

0 

0 

27 

33v 

13 

16 

.51 

.60 

.84 

75 

31 

41 

2 

3 

12 

16 

13 

16 

17 

23 

.41 

.53 

.77 

36 

8 

22 

0 

0 

10 

28 

8 

22 

10 

28 

.22 

.31 

.72 

39 

23 

59 

2 

5 

2 

5 

5 

13 

7 

18  , 

.59 

.72 

.82 

Completion  Rates: 


<M,C,D)/N  1 


II  (A,B,C,D)/N-W 


III  (A,8,C,D,X,Z)/N 
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Court*  Completion  lata* 
Fhoeali 

Totala  across  Hath  CourtM  and  Si 
Academic  Tear  1975-76 


tart 


\ 

Ttn  and       Comparison      Nuaber  of 


Credit 
(A.B.C.D) 


Failure 

(F) 


Grade  Dlat-  lbutlon 
Incomplete  Re-enroll 


(X) 


(2) 


Withdraw 

00 


Completion 
Bates 


*  Course 

Group 

Students 

M 

2 

N 

X 

N 

X 

N 

X 

N 

X 

I 

II 

m 

Fall  Semester: 

All  Courses 

Total 

1039 

355 

34 

9 

1 

12 

1 

368 

35 

o  295 

28 

.34 

.48 

.71 

TICCIT 

435 

71 

16 

0 

0 

5 

1 

268 

62 

91 

21 

.16 

.21 

.79 

Lacture 

604 

284 

47 

9 

1 

,  7 

1 

100 

17 

204*34 

.47 

.71 

Ka 

.65 

a  law 

Spring  Semester; 

All  Courses 

Total 

12053 

459 

38 

9 

1 

70 

6 

311 

26 

354 

29 

.38 

.54 

.70 

TICCIT 

476 

71 

15 

3 

1 

66 

14 

204 

43 

132 

28 

.15 

.21 

.72 

Lecture 

729 

388 

53 

6 

1 

4 

1 

107 

-JL  V  f 

15 

999 

JU 

.53 

.77 

.68 

nctnajLc  iears 

» 

Hath  007 

Total 

1056 

377 

36. 

13 

1 

33 

3 

336 

32 

297 

28 

.36 

.50 

.71 

TICCIT 

410 

58 

14 

2 

0 

25 

6 

218 

53 

107 

26 

;14 

.19 

.73 

Lecture 

646 

315 

49 

11 

2 

8 

1 

118 

18 

190 

29 

.49 

.69 

.68 

Hach  1061 

Total 

7363 

257 

35 

4 

1 

40 

5 

'227 

31 

207 

k 

.35 

.49 

.71 

TICCIT 

329 

57 

17 

0 

0 

39 

12 

153 

47 

80, 

/24 

.17 

.23 

.76 

Lecture 

407 

200 

49 

4 

1 

1 

0 

74 

18 

12/ 

31 

.49 

.71 

.68, 

Math  108 

Total 

3 

361 

114 

32 

1 

j 

0 

16 

4 

122 

34 

107 

30 

.32 

.45 

.70 

TICCIT 

161 

21 

13 

1 

1 

15 

9 

97 

60 

27 

17 

.13 

.16 

.83 

Lecture 

200 

93 

46 

0 

0 

1 

0 

25 

12 

80 

40 

.*i 

.77 

.59 

Math  1172 

Total 

297 

"146 

49 

2 

1 

24 

8 

46 

15 

79 

27 

.49 

.67 

.73 

TICCIT 

97 

22 

33 

b 

0 

21 

22 

24 

25 

30 

31 

.23 

.33 

.69 

Lecture 

200 

124 

62 

2 

1 

3 

v 

22 

11 

49 

24 

.62 

.82 

.74 

All  Courses. 

Total 

24503 

894 

36 

20 

1 

113 

731 

30 

690 

28 

.36 

,.51 

.71 

TICCIT 

997 

158 

16 

3 

6 

100 

10 

492 

49 

244 

24 

.16 

.21 

.75 

lecture 

1453 

736 

51 

17 

i 

13 

1 

239 

16 

446 

31 

.51 

.73 

.68 

Coaplatioa  Bates:  I 

(A,B,C,D)/N 

II 

(A,B,C,D)/N-W 

HI 

(A.B,C,D,l*Z)/B 

V 

^These  totals  Include  the  fall,  spring  and  summer  semesters* 

2 

Totals  for  Math  117  reflect  only  the  spring  and  gunner  semesters. 

*Tvo  students  audited  a  math  course  so  the  total  number  of  students  differs  by  two  with  the  t?tal  number 
of  grades.   Both  students  vers  In  lecture  classes  In  the  spring  semester,  one  in  Math  106,  and  another  in 
Math  108. 
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Credit    Failura    lacoaplata    la-anroll  Withdraw. 
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00 

N  X 
T  ~ 


Complatloa 
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Kath  31 

Total 
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59 

25 

40 

17 

0 

0 
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43 

37 

15 

.25 

.29 

.68 

T1CCIT 
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10 

9 

18 

16 

0 

0 

73 

63 

15 

19 

.09 

.10 

.71 
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rrogtMBM 

AA 

■ 

1 1 
11 

11 

A 
U 

0 

9A 

3d 
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1U 
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•H 
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79 

44 

56 

17 

22 

o 

0 

6 

8 

12 

15 

.5* 

•  6^ 
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L 
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Math  31 

215 

64 

30 

23 

15 

7 

78 

f  w 

36 

33 

16 
av 

.30 

.36 

TICCIT 

118 

29 

25 

11 

9 

11 

9  f 

55 

47 

12 

10 

.25 

.27 

.a 

Programed 

46 

13 

28 

6 

13 

3 

7 

16 

35 

8 

17 

.28 

.34 

.70 

Lecture 

51 

22 

43 

6 

12 

1 

2 

7 

14 

is 

29 

.43 

.61 

.59 

Kath  32 

TotU' 

76 

17 

22 

6 

a, 

0 

0 

36 

47 

17 

22 

.22 

.29 

.70 

TICCIT 

42 

2 

5 

6 

14 

0 

0 

28 

67 

6 

14 

.05 

.06 

.71 

Lecture 

34 

3.5 

44 

0 

0 

0 

.0 

8 

24 

u 

32 

.44 

.65 

.68 
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Hath  31 

Total 
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39 

23 

14 

8 

15 

9 

54 

31 

51 

29 

.23 

.32 

.62 

TICCIT 

57 

-  13 

23 

'2 

4 

11 

19 

20 

35 

'  11 

19 

.23 

.28 

.77 
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71 

14 

20 

7 

10 

2 

3 

22 

31 

26 

37 

.20 

.31 

.54 
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45 

12 

27 

5 

11 

2 

4 

12 

27 

14 

31 

.27 

.39 

.58 
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Total 

86 

32 

37 

8 

9 

7 

8 

27 

31 

12 

14 

.37 

.43 

.77 
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42 

6 

14 

3 

7 

7 

17 

19 

45 

7 

17 

.13 

.17 
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44 

26 

59 

5 

11 

0 

0 

8 

18 

5 

11 

.59 

.67 
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Total 

627 
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26 

77 

30 

5 
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37 
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.32 

.68 
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52 

18 

31 

22 

8 
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51 

38 

13 

.18 

.21 

.76 
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32 

20 

18 
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5 

3 

62 

39 

44 

27 
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.27 

.61 

Lactura 
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78 

45 

28 
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3 

2 

25 

14 

41 

23 
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.58 

.61 

Total 
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49 

30 

14 

9 
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63 

39 

29 

18 
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.73 

TICCIT 

84 

8 
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5 
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8 

47 
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13 
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5 

9 

12 

10 
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.66 

.73 
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1000  students  enrolled  for  math  courses  with  the  TICCIT  program,  Phoenix 
provides  a  stronger  test  case  for  examining  TICCIT' s  effects  on  course  comple- 
tion.   And  the  effect  Vas  dramatic.    For  the  first  course  listed  in  these 
tables,  Phoenix's  Math  007  in  the  fall  semester  of  1975,  only  27  of  the  215 
students  who  had  registered  for  the  course  under  the  l^Cyp condition  actually 
received  grades  high  enough  to  carry  credit.    This  completion  rate  of  13Z 
stands  in  sharp  contrast  to  the  lecture  rate  of  47%.    Yet  the  course  and  term 
of  this  example  represent  the  highest  enrollment  and  the  best  control  in  the 
evaluation.     It  behooves  us  to  take  a  closer  look  at  the  fall  results  for  this 
introductory  algebra  course.  J$T/:  ^ 

Several  plausible  explanations  aside  from  a  treatment  effect  might  be 
offered  as  alternative  causes  of  the  sizeable  gap  in  completion  rates  between 
the  TICCIT  and  lecture  conditions.    Let  us  try  to  consider  some  of  them  now 
and  hold  others  until  coming  to  the  analyses  for  completion.    One  prominent 
explanation  might  be  the  use  of  an  inappropriate  procedure  in  calculating 
completion  rates.    There  seems  no  better  source  than  official  class  rosters 
so jue  must  turn  to  other  interpretations  of  completion.    Perhaps  withdrawals 
unfairly  affected  the  TtCCIT  condition  and  an  adjustment  for  attrition  would 
ease  the  difference.    The  second  type  of  completion  rate  takes  an  approximate 
correction  for  attrition  into  account:     it  deducts  the  number  of  withdrawals 
from  total  enrollment.     But  the  completion  rates  under  this  interpretation 
become  16Z  for  the  TICCIT  condition  and  68Z  for  lecture  classes.  Corrections 
for  attrition  seem  to  exacerbate  xather  than  ease  the  difference. 

The  problem  may  lie  instead  with  our  insistence  on  a  fixed  time  constraint 
for  a  self-paced  program.     If  we  were  to  relax  this  constraint  and  anticipate 
completions  beyond  the  arbitrary  deadline  of  an  academic  term,  rates  under 
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the  Ticcit  condition  would  improve.    The  high  percentage  of  reenroll  grades 
in  the  TicciT  condition  suggests  that  there  is  merit  to  this  position.  For 
Math  007  in  tne  fall  semester  of  1975  reenroll  grades  constituted  63X  of  the 
TICCIT  grade  distribution  and  18%  of  the  distribution  for  the  lecture  condition. 
Th«  third  type  of  completion  rate  reflects  this  fact.    When  we  consider  re;- 
enrolls  and  inCompletes  as  eventual  completions.,  the  rate  changes  to  favor 
the  TICCIT  program.    The  figure  for  TICCIT  classes  is  77*  as  compared  to 
68Z  for  traditional  classes.    At  lea8t  in  terms  of  attendance  from  the  start 
to  the  conclusion  of  an  academic  term  there  is  a  slight  advantage  to  the 
TICCIT  prograp  over  classroom  instruction.    To  take  this  argument  a  step 
further  and  impiy  a  similar  advantage  in  terms  of  completing  a  course  with 
credit  would  be  unwarranted.  <- 

Xt  cannot  be  assumed  that  all  students  who  receive  a  grade  equivalent  to 
one  that  necessitates  subsequent  enrollment  in  order  to  receive  credit  do 
both  later  enroll  and  earn  course  credit.    Of  the  136  students  with  Z  grades 
from  TIccit  sections  of  Math  007  from  the  fall  semester.  58  students  enrolled 
for  Math  007  again  in  the  spring  term  (see  Appendix  N  on  subsequent  term 
enrollme^  #    Already ^|he  potential  for  eventual  completion  has  reduced  the 
"°8t  °Ptimi8tic  fall  rate  to  about  40Z  instead  of  77*.     It  is  even  less  since 
only  31  students  completed  Math  007  under  the  TICCIT  program  in  the  spring 
semester,     SinCe  nearly  every  student  who  did  reenroll  followed  the  same 
instruction^  condition,  a  "best  case"  estimate  of  completing  introductory 
algebra  aftir  enrolling  in  the  course  for  a  second  term  would  be  31  of  58 
students  Qtf  slightly  better  than  a  student's  probability  of  completing  the  same 
course  within  one  term  in  a  reguiar  classroom  format.    But  only  10  of  these 
31  students  were  actually  repeating  the  course.     So  the  completion  rate  among 
students  reenrolling  for  introductory  algebra  in  the  spring  semester  was  ,17% 
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(10  grades  with  'credit  for  58  students)  and  it  was  15%  for  students  enrolling 
in  the  TICCIT  program  for  the  first  time  (21  grades  with  credit  for  137  students). 
The  close  similarity  in  completion  rates  among  students  repeating  courses  and 
those  enrolling  for  the  first  time  will  be  seen  again  in  the  analyses  for  this 
prograto  outcome* 

As  long  as  completion  is  viewed  as  the  receipt  of  a  course  grade  carrying 
college-  credit,  the  rate  for  th*  TICCIT  program  remains  well  below  that  of 
lecture  classes.     When  adjusted  for  attrition  the  completion  rates  between 
conditions  differ  by  an  even  greater  margin.     The  primary  reason  for  this 
difference  is  the  much  larger  group  of  students  who  remain  in  a  TICCIT  class 
but  fail  to  satisfy  course  requirements  within  an  academic  term.     Yet  when, 
these  students  take  the  same  course  in  a  subsequent  term,  their  rate  of  comple- 
tion after  a  second  term  seems  hardly  better  than  that  of  students  enrolled  for 

4 

the  first  time.     Only  if  we  accept  attendance  at  the 'beginning  and  end  of  a 

l 

term  as  sufficient  for  completing  a  course  does  the  rate  for  the  TICCIT  program 
exceed  that  for  the  familiar  lecture  format.     Otherwise  the  mechanics  of 
computing  completion  rates  seem  not  to  offer  an  explanation  of  the  difference 
between  conditions. 

Perhaps  the  cause  underlying  the  difference  observed  in  Math  007  for 

the  fal I,  semester  relates  to  the  nature  of  the  TICCIT  program  for  that  course, 

« 

term  and  college.     The  curricular  materials  for  TICCIT' s  introductory  algebra 
might  be  weaker  than  those  that  correspond  to  intermediate  or  advanced  algebra. 
But  the  rates  for  the  TICCIT  program  in  Math  106  and  Math  117  (see  Tabled. 2) 
show  the  same  relationship  between  conditions.     Completion  does  improve  for  the 
most  advanced  math  course,  however  this  happens  for  the  lecture  condition  as 
well  as  the  TICCIT  program.     Differences  in  the  quality  of  TICCIT  curricular^? 
materials  across  courses  is  not  a  tenable  explanation.     The  design  of  the 
program  was,  after  all,  constant  across  applications. 
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Similarly,  we  can  rule  out  idiosyncracies  due  to  term  or  college.  The 
TICCIT  system  was  at  its  most  unstable  stage  in  the  fall  semester  of  1975 
compared  to  later  terms  of  the  demonstration  period.    The  computer  system 
seemed  prone  to  crashes  and  the  course  materials  in  need  of  revision.  There 
were  vast  improvements  in  system  reliability,  and  courseware  revision  became  a 
continuing  process.     Figure  4.1  traces  TICCIT  completion  rates  for  Math  007 
from  the  fall  of  1974  to  the  fall  of  1976.    This  two-year  period  spans  the 
initial  implementation  of  the  program  at  Phoenix  and  the  college's  ovm  use  of  { 
the  system  after  the  cessation  of  our  data  collection  for  math  courses.  There 
is  an  increase  in  completion  rate  over  time  but  the  results  for  the  TICCIT 
program  still  fall  far  short  of  those  obtained  in  lecture  classes. 

The  average  completion  rate  for  Alexandria's  Algebra  «I  is  remarkably 
similar  to  that  for  Phoenix's  Beginning  Algebra.     For  the  1975-76  academic  year 
the  completion  rates  for  Math  31  were  18%  for  TICCIT  classes  and  45%  for 
lecture  classes  while  those  for  Math  007  were,  respectively,  14%  and  49%.  This 
similarity  dismisses  another  plausible  explanation  for  the  Math  007  results  in 
the  fall  term:     the  disparity  in  rates  between  conditions  seems  not  to  be  a 
function  of  the  specific  college.    The  comparable  results  across  colleges  is 
instructive  also  because  Phoenix  followed  the  concept  of  mainline  instruction 
with  a  high  student-teacher  ratio  while  Alexandria  adhered  to  regular  class 
sizes  for  all  conditions  (see  Appendix  M) .     So  the  student- teacher  ratio  for 
the  TICCIT  program  had  scant  impact  on  completion  rates. 

What  plausible  alternatives  remain  for  explaining  differences  in  comple- 
tion rates  between  conditions?    The  difference  in  our  original  example,  Math 

f 

007  in  the  fall  term,  might  be  attributed  to  instructors.    Except  a  student  on 
the  TICCIT  system  had  no  instructor  other  than  TICCIT.    Members  of  the  math 
department  shared  supervisory  duties  for  TICCIT  classes  by  rotating  according 

J 
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to  an  hourly  schedule.  There  were  no  course  sections  as  such  and  so  there 
could  be  no  effect  on  a  student's  grade  due  to  an  instructor.  Half  of  the 
math  department  at  Phoenix  shared  responsibility  for  students  on  the  TICCIT 
system.  Then  perhaps  instructors  for  lecture  sections  were  overly  lenient 
in  grading,  or  those  responsible  for  TICCIT  classes  overly  stringent.  The 
evidence  to  counter  such  arguments  appears  in  Chapter  5:     Student  Achievement. 

Only  dissimilar  student  groups  under  each  condition  seems  to  be  left 

> 

as  an  alternative  explanation.     But  recall  that  students  were  randomly  assigned 
to  conditions.    And  analyses  for  completion  take  prior  ability  and  student 
background  into  account.     The  consistency  of  the  completion  figures  across 

courses,  terms  and  colleges  is  simply  too  high    for  us  to  entertain  rival 

i 

explanations.     Lower  completion  i*ates  were  part  of  the  TICCIT  program  for 
math.     It  is  natural,  then,  to  wonder  what  caused  this  effect.    The  question 
is  appropriate  but  an  answer  would  be  premature  before  presenting  rates  from 
English  courses  and  results  of  analyses. 

At  least  another  point  of  consistency  in  completion  results  should  be 
noted  for  math  courses.     The  rates  for  finishing  a  course  with  credit  run 
closely  together  for  TICCIT  classes  and  programmed  Instruction  throughout 
Alexandria's  1975-76  academic  year  (see  Table  4.3).     The  audio-tutorial 
format  for  Phoenix's  Beginning  Algebra,  abandoned  after  the  spring  qf  1975, 
had  an  average  completion  ratfc  of ^2  for  the  1972-73  academic  year.  Although 
all  compare  unfavorably  with  the  completion  rates  found  in  lecture  conditions, 
the  completion  results  from  self-paced  programs  appear  fairly  consistent 
between  one  another. 
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j  4.3.b  English  Courses 

Tables  4.4  and  4,5  present  grade  distributions  and  completion 
rates  for  English  courses  at  Phoenix  and  Alexandria,    Courses  from  Phoenix 
again  represent  a  stronger  test  case  for  the  TICCIT  program,  A  larger  number  of 
students  took  English- courses  on  the  TICCIT  system  and  these  courses  adhered 
more  closely  to  the  concept  of  mainline  computerrassisted  instruction.    But  the 
applications  at  the  two  colleges  were  also  quite  different.    English  019  and 
English  D29  serve  as  prerequisites  to  theThoenix  course  equivalent  to  college 
composition.    These  courses  cover  the  fundamentals  of  English  for  the  student 
weak  in  writing  skills.    In  contrast,  the  course  at  Alexandria  is  its  English 
Composition  I.  Wstructors  there  used  the  TICCIT  program  primarily  in  those 
course  sections  that  met  five  rather  than  the  usual  three  periods  per  week. 
And  such  laboratory  sections  met  in  traditional  classrooms  for  lecture- 
discussion  sessions  as  well  as  working  on  the  TICCIT  system.    The  Alexandria 
application  combined  TICCIT  with  regular  teaching  practices  and  so  represents 
the  use  of  the  TICCIT  program  in  an  adjunctive  manner.    The  context  of  TICCIT  s 
application  at  the  two  colleges  is  a  necessary  introduction  to  the  tables  on 
completion. 

Obviously  the  completion  rates  for  the  TICCIT  condition  were  higher  in 
English  courses  than  they  had  been  in  math.    The  lowest  rate  for  a  TICCIT 
application  in  an  English  course  is  higher  than  any  attained  in  a  math  course. 
It  seems  that  regardless  of  condition  the  proportion  of  students  who  receive 
course  credit  is  higher  for  English  courses.    There  is,  still,  a  gap  between 
the  TICCIT  and  lecture  conditions.    But  the  extent  of  the  difference  is  much 
smaller.    At  Phoenix  fewer  than  twenty  percentage  points  separate  the  rates  for 
the  two  conditions.    For  Alexandria  the  size  of  the  difference  stays  at  about 
ten  percentage  points  across  terms* 
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Course  Completion  Rates 
Phoenix 
Englieh  019  and  029 
Calendar  Year  1976  < 


Grade  Distribution 


Term  and 
Course 

Spring  Semester: 
English  019 


English  029 


All  Courses 


Summer  Semester: 
English  029 


Fall  Semester: 
English  019 

English  029 

All  Courses 

Across  Semesters: 


Comparison 
Group 


Number  of 
Students 


Credit 
(AfBfCfD) 


^Failure 
N  Z 


Incomplete 

'  (X) 


Re-enroll 

(Z) 


Total 

244 

113 

46 

1 

0 

11 

5 

42 

TICCIT 

88 

30 

34 

0 

0 

9 

10 

22 

Lecture' 

156 

83 

53 

1 

1 

2 

1 

20 

Total 

138 

86 

62 

7 

5 

0 

0 

23 

TICCIT 

64 

33 

52 

0 

0 

0 

0 

13 

Lecture 

74 

53 

72 

7 

9 

0 

0 

10 

Total 

382 

199 

52 

8 

2 

11 

3 

65 

TICCIT 

152 

63 

41 

0 

0 

9 

6 

35 

Lecture 

230 

136 

59 

8 

3 

2 

1 

30 

Total 

100 

61 

61 

1 

1 

3 

3 

24 

TICCIT 

63 

37 

59 

1 

2 

3 

5 

13 

Lecture 

37 

24 

65 

0 

0 

0 

0 

11 

Total 

TICCIT 

Lecture 

Total 

TICCIT 

Lecture 


Total 

TICCIT 

Lee9ure 


30 


407 

258 

63 

2 

0 

24 

6 

51 

13 

175 

93 

53 

0 

0 

13 

7 

34 

19 

232 

165  - 

71 

2 

1 

11 

5 

17 

7 

216 

129 

56 

13 

6 

24 

11 

17 

8 

111 

54 

49 

3 

3 

24 

22 

2 

2 

105 

66 

63 

10 

10 

0 

0 

s  *15 

14 

623 

378 

61 

15 

2 

48 

8 

68 

11 

286 

147 

51 

3 

1 

37 

13 

36 

13 

337 

231 

69 

12 

4 

11 

3 

32 

9 

Withdraw 
(W) 


Completion 
Rates 


N 

Z 

I 

II 

III 

77 

32 

.46 

.68 

.68 

31 

.34 

.49 

.69 

50 

3?v 

•  j  j 

•  fO 

£7 

>> 

*t  .62 

.74 

.79 

■UK 

KM* ' 

.52 

.72 

.72 

4  1 

•  72 

.85 

99/ 

26 

.52 

.70 

12. 

45 

30 

.41 

.59 

.7b 

54 

23 

.59 

.77 

.73 

11 

11 

.61 

.69 

.88 

9 

14 

.59 

.69 

.84 

2 

5 

.65 

.69 

.95 

71 

17 

.63 

.77 

.82 

34 

19 

.53 

.66 

.80 

37 

16 

.71 

.85 

.83 

42 

19 

.56 

.69 

.75 

28 

25 

.49 

.65 

'ZSel 

14 

13 

.63 

.73 

L13 

18 

.61 

.74 

62 

22 

.51 

.66 

.77$ 

51 

15 

.69 

.81 

.61 

English  019 

Total 

TICCIT 

Lecture 

651 
263 
388 

371 
123 
248 

57 
47 
64 

3 
0 
3 

0 
0 
1 

35 
22 
13 

5 
8 
3 

93 
56 
37 

14 
.21 
10 

148 
*1 
87 

23 
23 
22 

.57 
.47 
.64 

.74 
.61 
.82 

.77 
.76 
.77 

English  029 

Total 

TICCIT 

Lecture 

454 
238 
216 

267 
124 
143 

59 
52 
66 

21 
4 
17 

5 
2 
8 

27 
27 
0 

6 
11 

0 

64 
28 
36 

14 
12 
17 

'  75 
55 
20 

17 
23 
9 

.59 
.52 
.66 

.70 
.68 
.73 

.79 
.75 
.83 

English  019  &  029 

Total 

TICCIT 

Lecture 

1105 
v  501 
604 

638 
247 
.  391 

57 
49 
65 

24 
4 
20 

2 
1 

3 

62 
49 
13 

6 
10 

2 

157 
84 
73 

14 
17 
12 

223 
116 
107 

20 
23 
18 

.57 
.49 
.65 

.72 
.64 
.79 

.78 
.76 
.79 

Completion  Rates: 


(A,B,C,D)/N 


II  (AtB,CtD)/N-W 


III 


(AfB>C>D,XfZ)/N 


Table  4.4 
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Course  Completion  Rates 
Alexandria 
English  111 
Calendar  Year  1976 


Term  and 
Course 

Winter  Quarter: 


Comparison  Number  of 
Group  Students 


Credit 
(M.C.D) 

N  Z 


Failure 

m 

N  I 


Grade  Distribution 

Incomplete  Re-enroll 
(I) 


N  't> 
-  1 

/ 


(R) 
l"  Z 


Withdraw 
(W) 

N  Z 


Completion, 
Rates  p 

I  It 


'III* 


English  111 

Total 

211 

114 

54  ' 

8 

4 

14 

7 

12 

6 

63 

30 

.54, 

.77 

.66 

TICCIT . 

in 
ii/ 

JO 

jU 

U 

(J 

11 
11 

Q 
J 

L 
0 

c 
J 

Hi 

•  jU 

.// 

,01 

Lecture 

94 

56 

60 

o 
O 

n 

9 

3 

3 

0 

t 
O 

01 

Zl 

a 

.60 

10 
,10 

.W 

Spring  Quarter: 

English  111 

Total 

339 

243 

72 

9 

3 

7 

2 

15 

4 

63 

19 

.72 

.86 

' ,  .78 

TICCIT 

108 

70 

65 

2 

2 

4 

4 

12 

11 

19 

18 

,65 

.79* 

.80 

Lecture 

231 

173 

75 

7 

3 

3 

1 

3 

1 

44 

19 

.75 

•93  • 

'  .'77 

Fall  Quarter: 

English  111 

Total 

300 

200 

67 

32 

11 

9 

3  ■ 

13 

4 

46 

15 

.67 

.  79: 

.7* 

TICCIT 

161 

102 

63 

20 

12 

5 

3 

12 

7 

22 

14 

.63 

.73, 

♦73 

Lecture 

139 

98 

71 

12 

9 

4 

3 

1 

1 

'  24 

17 

.71 

.85 

.74 

Across  Quarters: 

English  111 

Total 

850 

55f 

k 

49 

6 

30 

4 

40 

5 

172 

20 

.66 

.82 

.74 

TICCIT 

386 

240 

62, 

22 

6 

20 

5 

30 

8 

83 

22 

.60 

,76 

.73 

Lecture 

464 

317 

68 

27 

6 

10 

2 

10 

2 

89 

19 

.70 

\B7 

.75 

Completion  Rates:      I  (A,B,C,D)/N 


II  .(A,B,C,D)/H-W 


III  (A,B,C,D,I,R)/H 
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Generally  the  withdrawal  rate  in  English  courses  stayed  about  the  same 
whether  classes  spent  time  on  the  TICCIT  system  or  not.    Hence  completion  rates 
adjusted  for  attrition  should  reflect  a  difference  between  conditions  compara- 
ble to  that  shown  by  our  preferred  definition.  The  summaries  for  the  entire 
year  best  illustrate  this  point.    The  magnitude  of  the  difference  exhibited  at 
Phoenix  across  terms  and  courses  is  sixteen  percentage  points  when  we  take 
completion  as  the  ratio  of  grades  with  credit  to  total  enrollment:     49%  for 
TICCIT  classes  and  65%  for  lecture-discussion  classes.     It  does  not  change  the 
difference  to  adjust  these  rates  for  attrition:     64JK  and  79%  still  ref:||&t  a 
fifteen-point  gap.    For  Alexandria  the  size  of  the  difference  also  remains 
stable.     It  is  ten  percentage  points  for  completion  as  receipt  of  course 
credit,  60%  as  opposed  to  70%,  and  eleven  percentage  points  for  completion 
corrected  for  attrition,  76%  contrasted  with  87%.    The  direction  of  the 
difference  always  favors  lecture-discussion  classes. 

Two  exceptions- occurred  to  completion  rates  otherwise  unaffected  by 
withdrawal.    One  was  English  029,  a  course  offered  by  the  evening  division 
of  the  College  and  staffed  largely  by  part-time  instructors.    The  other 
was  English  111  in  the  winter  quarter  of  1976;  that  was  the  first  quarter 
in  which  Alexandria  had  several  classes  on  the  TICCIT  system  and  instructors 
were  yet  finding  their  way  through  the  English  courseware.     In  these  cases 
correcting  completion  rates  for  withdrawal  jioea  narrow  the  difference  between 
conditions.     It  becomes  negligible  for  English  029  in  the  spring  and  summer 
semesters  and  English  111  in  the  winter  quarter.    Although  these  instanced  seem 
confounded  with  either  part-time  faculty  practices  or  program  novelty,  they  do 
raise  the  possibility  that  students  who  attend  classes  stand  approximately  the 
same  chances  for  receipt  of  course  credit  under  either  instructional  condition. 
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If  we  accept  attendance  as  evidence  of  completion  or  assume  students 
eventually  fulfill  course  requirements  left  outstanding  at  the  end  of  an 
academic  term,  it  is  then  appropriate  to  turn  to  the  third  type  of  completion 
.rate.    Now  grades  of  incomplete  and  reenroll  as  well  as  those  with  course 
dredit  count  towards  the  total  number  of  completions.    And  grades  without 
pjenalty  or  credit  were  usually  about  twice  as  frequent  in  TICCIT  classes 
as  in  lecture-discussion  classes.    We  would  therefore  expect  such  a  generous 
interpretation  of  completion  to  bring  the  conditions'  rates  closer  together. 
It  does. 

Consider  the  complement  of  those  courses  affected  by  corrections  for 
withdrawal:     English  019  at  Phoenix  and  English  111  in  the  spring  and  fall 
quarters  at  Alexandria.     For  the  calendar  year  the  completion^ rates  for 
English  019  become  76%  for  TICCIT  and  77%  for  lecture.     Sections  of  English 
.111  on  the  TICCIT  system  in  the  spring  quarter  yield  a  slightly  higher  rate 
than  lecture  sections ,  80%  and  78%  respectively;     In  the  fall  quarter  there 
is  just  one  percentage  point  separating  the  two  conditions'  outcomes. 

At  best,  under  a  most  lenient  definition  of  completion,  the  completion 
rates  for  TICCIT  and  lecture  classes  appear  comparable.     But  this  assumes 
all  students  with  grades  of  incomplete  or  reenroll  eventually  receive  course 
credit.    The  math  results  suggest  such  an  assumption  is  unfounded  and  our 
analyses  confirm  this.     So,  unless  attendance  is  sufficient  as  a  program 
outcome,  the  TICCIT  program  leads  to  lower  completion  rates  in  English  courses 
just  as  it  did  in  math. 

In  several  cases  adjusting  the  completion  rates  for  withdrawals  notably 

i  ■ 

lessens  the  difference  between  conditions.     However^  these  cases  also  involve 
llovelty  in  the  form  of  part-time  instructors  teaching  classes  or  regular 
faculty, experimenting  with  TICCIT  use.     Inconsistent  grading  practices  seem  to 
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reflect  this  novelty  (e.g.,  notice  that  the  percentage  of  reenrolls  doubled 
between  the  winter  and  spring  quarters  while  withdrawals  dropped  by  half) 
and  stand  as  competing  explanations  for  the  lessening  of  differences  in  rates 
corrected  for  withdrawals. 

On  the  whole  it  seems  reasonable  to  conclude  that  in  English  courses  as 
in  math  the  T1CC1T  program  causes  a  decrease  in  completion  rates  in  comparison 
to  lecture^disbvssion  classes.     Such  a  conclusion  needs  to  be  tested.    That  is 
the  purpose  of  the  next  section  of  this  chapter.     Perhaps,  though,  two  points 
should  be  briefly  reiterated  before  turning  to  the  analysis  of  TICCIT's  impact 

i 

on  completion.     Both  concern  relative  completion  rates  among  applications  of  the 
T1CC1T  program.    The  first  is  that  the  lowest  completion  rate  for  T1CC1T  sections 
of  an  English  course  was  higher  than  any  attained  for  the  T1CC1T  program  in  a 
mat^ course.    The  second  is  that  the  smallest  difference  between  conditions  for 
English  courses  arose  with  the  adjunctive  use  of  T1CC1T. 

I*. 4    Analysis  of  ProRram  Effects  \ 

The  purpose  of  our  data  analysis  is  to  establish  whether  an  apparent 
difference  in  outcomes  was  due  to  the  curricular  program  under  study  and,  if 
there  were  treatment  effects,  estimate  tHfe  extent  of  the  program's  impact  on 
student  performance.  Here  the  specific  outcome  of  interest  is  course  completioj? 
defined  as  the  receipt  of  a  final  grade  carrying  college  credit.     In  order  to 
establish  a  difference  in  results  as  a  treatment  effect  we  must  somehow  isolate 
a  program's  impact  and  take  other  plausible  explanations  into  account.  Our 
approach  to  this  problem  was  to  base  the  evaluation  on  comparisons  between 
student  groups  exposed  to  either  the  TICCIT  program  or,  as  a  relative  standard, 
lecture-discussion  instruction.    Also  data  were  collected  on  probable  correlates 
of  student  performance  such  as  entrance  knowledge  of  the  subject  matter  and 
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grade  point  average.    By  taking  these  factors  into  consideration  and  either 
assigning  students  randomly  to  conditions  or  adjusting  estimates  for  students 
similar  across  conditions,  we  hoped  to  isolate  treatment  effectfc. 

Generally  the  population  of  students  that  concerns  us  in  examining  comple- 
tion  is  that  which  registers  for  a  course  at  the  beginning  of  an  academic  term. 
Where  natural  divisions  exist  among  classes,  such  as  day  and  evening  or  labora- 
tory (five  sessions  per  week)  and  regular  (three  sessions  per  week),  these 
define  subpopulations .     Student  characteristics  might  also  define  .slibpopulations. 
But  we  need  not  specify  a  different  treatment  effect  for  every  particular 
group  in  the  population.    The  only  ones  warranting  separate  estimates  will 
be  those  for  which  there  is  evidence  of  a  differential  treatment  effect.  When 
the  impact  of  the  curricular  program  varies  according  to  q)ome  independent 
variable,  then  it  is  appropriate  to  estimate  effects  for  the  subgroups  defined 
by  that  variable. 

It  is  difficult  to  conceive  of  a  differential  treatment  effect  in  general 
terms.    The  most  common  example  is  probably  a  compensatory  program  that  benefits 
some  students  more  than  others:     those  weakest  in  entrance  knowledge  might 
§how  the  largest  gains  in  achievement.     Such  a  differential  effect  on  achievement 
would  be  seen  as  an  interaction  between  treatment  condition  and  pretest  in 
our  analyses.     But  let  us  also  interject  an  example  taken  from  our  analyses 
of  completion  in  math  courses.    The  TICCIT  program  in  algebra  required  that 
students  do  nearly  all  their  course  work  on  the  computer  system.  Part-time 
students  with  responsibilities  at  home  or  work  might  find  such  mainline  instruc- 
tion inconvenient  and  gradually  fall  behind  in  their  stucfies.    This  is  a 
plausible  interpretation  of  what  happened  in  Beginning  Algebra  at  Phoenix. 
Possibly  because  of  an  inability  to  study  outside  of  school,  part-time  students 
as  a  group  hd^^lower  completion  rate  than  full-time  students  in  the  TICCIT 
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program*    This  Interaction  between  treatment  condition  and  student  status  calls 
for  separate  estimates  of  treatment  effects:     one  for  part-time  students  and 
another  for  full-time  students  (see  Tables  4*6  and  4.10). 

Interactions  of  the  TICCIT  condition  with  other  variables  assume  Importance 
In  our  analysis  of  completion  for  two  reasons*     The  first  relates  to  their 
possible  Influence  on  other  aspects  of  the  evaluation,  particularly  the  study 
of  student  achievement  and  attitudes*     If  part  of  TICCIT's  effect  on  completion 
Is  to  exclude  or  Include  certain  student  groups  from  the  posttestlng  or  survey- 
ing conducted  at  the  time  of  course  completion,  there  might  be  a  systematic 
bias  In  later  comparisons  of  achievement  and  attitude  results*     This  threat  to 
validity  holds  only  where  one  condition,  whether  TICCIT  or  lecture,  exerts  a 
differential  effect  on  completion*     But  aside  from  possible  negative  consequences 
for  the  evaluation  there  is  valuable  policy  information  to  be  gleaned  from 
interactions*     If  a  school  knows  which  students  benefit  most  from  alternative 
curricular  programs,  it  can  begin  to  assign  students/to  programs  based  on  that 
information  and  perhaps  b^gitv  to  Improve  overall  results*     Or  such  interactions 
may  suggest  different  and  hopefully  more  Effective  applications  for  a  new  program 

For  these  reasons  interactions  will  be  presented  as  part  of  our  analyses 
whenever  they  constitute  a  significant  contribution  to  the  explanation  of 
an  outcome's  variance*     If  the  t-statistic  associated  with  an  interaction 
surpasses  a  value  of  approximately  2*00,  that  interaction  accounts  for  outcome 
variance  left  unexplained  by  our  other  independent  variables  and  therefore 
becomes  part  of  the  regression^analysis  through  a  step-wise  procedure*  We 
actually  tested  a  large  number  of  potential  interaction  terms  in  our  regressions* 
Some  might  be  expected  to  enter  the  analyses  by  chance  alone*     Rather  than 
ignore  these  as  simply  chance  occurrences,  however,  we've  chosen  to  present  all 
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such  terms  in  our  tables •    This  represents  a  conservative  position  in  its 
explicitness  about  threats  to  validity  and  a  liberal  one  in  its  explication 
of  results  with  policy  implications.     So  it  is  appropriate  to  exercise  caution 

before  taking  any  single  interaction  as  a  definitive  statement  for  school 

f' 

practices.     instead  only  those  replicated  across  terms  and  colleges  should 
be  taken  as  conclusive^ 

Of  course  establishing  response  functions  with  single-term  independent 
variables  alway8  preceded  our  tests  for  differential  effects  (see  Section  2.4, 
Data  Analy8i8  in  Chapter  2:     Methdds  and  Measurement).    The  .separate  response 
functions,  one  for  each  condition,  could  actually  be  simplified  to  a  regression 
model.     Unless  significant  interactions  occurred,  the  estimated  treatment 
effect  is  giVen  by  just  one  term  in  the  model.     The  tables  that  follow  refer 
to  this  term  as  the  independent  variable  labeled  "Condition".    Wherever  an 
interaction  ^  listed  as  part  Qf  the  analysis,  there  is  no  single  treatment 
effect,  an(j  several  estimates  corresponding  to  subpopulations  will  be  presented. 
The  dependent  variable  in  all  analyses  for  completion  was  the  final  course 
grade  reduced  to  two  values:     l  if  the  student  received  course  credit  (grades  of 
A,B,C,  or  o  in  most  courses  and  S  in  developmental  math  at  Alexandria  or  P  in 
special  cases)  and  0  if  the  8tudent  fails  to  fulfill  course  requirements 
within  an  academic  term  (grades  of  F,W,X  or  Z  at  Phoenix  and  F,W,I  and  R  at 
Alexandria  a8  y^ll  as  U  in  developmental  math). 

4-4. a  Math_C°ur8e8_  ^ 

Tables  in  this  section  present  the  analyses  for  completion  as  a 
program  Outcome  in  math  courses.     It  is  naturally  tempting  to  proceed,  immedi- 
ately to  the  statistical  tests  but  a  few  general  remarks  should  be  made 
first.     Each  course  appears  along  with  a  simplified  regression  model  for  its 
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completion  rate.     We  consider  each  course  separately,  consistent  with  our 
Interest  In  student  population  enrolled  for  a  course  and  our  use  of  course- 
specific  achievement  measures.     Multiple  courses  and  terms  provide  an  oppor- 
tunlty  to  study  replications  of  effects. 

,  The  model  reported  for  each  course  and  term  draws  on  data  from  a  student 

V 

sample  which  Is  always  less  than  the  total  enrollment.     This  Is  because  not  all 


students  who  registered  for  a  course  actually  attended  classes  and  sometimes 
there  were  exceptions  to  our  plan  of  collecting  data  in  all  sections.     Also  as 
the  number  of  variables  in  the  vector  of  covariates  increased,  the  chances  of 
omitting  a  student  due  to  a  single  missing  value  for  an  independent  variable 
increased.  But  incomplete  data,  occasional  disruption^  in  data  collection  and 
attendance  do  not  account  for  the  low  sample  size  in  Phoenix's  Beginning  " 
Algebra  for  the  spring  semester  and  Alexandria's  Algebra  I  in  the  winter  and 
spring  quarters.     In  these  cases  a  large  percentage  of  students  simply  avoided 
taking  pretests  and  did  so  with  some  Justification.     These  students  had  already 
taken  the  pretest   in  a  prior  term  and  therefore  objected  to  a  second  administra- 
tion when  they  enrolled  for  the  same  course  again.     At  Phoenix,  for  example, 
over  three-quarters  of  the  students  who  reenrolled  for  Math  007  on  the  TICCIT 
system  avoided  a  second  pretest  administration.  The  percentage  pretested  was 
obviously  much  higher  for  students  enrolled  in  a  course  for  the  first  time. 

The  number  and  nature  of  independent  variables  included  in  the  analyses 
follow  fjrom  three  considerations.     When  students  were  randomly  assigned  to 
conditions  (the  fall  semester  at  Phoenix),  there  was  no  need  for  an  extensive 
base  of  covariates.     The  assignment  procedure  itself  was  a  reasonable  guarantee 
of  student  similarity  across  conditions.     But  when  the  evaluation  relied  on 


quasi- random  assignment,   like  blind  section  selection,  or  on  straightforward 
student  preference,  as  was  the  case  with  English  courses  at  Phoenix,  there  is 
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cause  to  check  further  on  student  similarity.    The  variables  in  out  models  do 
not,  however,  represent  the  full  set  available  to  us  (see  Figure  2.7).  Those 
omitted  were  found  to  be  unrelated  to  completion  and  only  further  reduced 
sample  size  due  to  incomplete  data.    Indeed  we  might  have  anticipated  that 
pretest  score  would  override  any  contribution  to  the  model's  power  by  high 
school  or  grade  point  average.    Such  facts  were  established  from  preliminary 
work  with  data  from  the  implementation  trials  and  confirmed  by  exploratory 
analyses  for  the  demonstration  period.    Finally,  where  the  simple  n^odel 
could  be  significantly  improved  by  the  addition  of  an  interaction  term,  that 
term  was  also  included  in  the  regression  model. 

The  results  of  these  analyses  appear  in  Tables  4.6  and  4.7  for  math 
courses  at  Phoenix  and  Alexandria,  respectively.    As  might  be  expected  from 
the  data  summaries  on  course  completion,  the  consistency  of  the  results  across 
courses  and  terms  is  striking.    The  TICCIT  program  had  a  significant  negative 
effect  on  student  completion  of  course  requirements  within  an  academic  term. 
This  holds  for  all  seven  analyses  conducted  for  Phoenix  math  courses  and  three 
of  the  five  done  for  Alexandria.    And  there  is  reason  to  question  not  the  ten 
significant  cases  but  the  two  exceptions. 

The  regression  weights  given  for  TICCIT  as  the  instructional  condition 
correspond  to  the  estimated  treatment  effect  (t) .    Thus  the  estimated  effect 
of  the  TICCIT  program  on  completion  for  Math  007  In  the  fall  semester  was 
-.359.    This  corresponds  closely  to  the  actual  difference  in  completion  rates 
between  TICCIT  and  lecture  classes,  -.34  (see  Table  4.£),  calculated  directly 
from  official  college  rosters.    Even  the  weight  for  Math  31  in  the  winter 
quarter,  -.176,  matches  the  actual  difference  in  completion  rates,  -.18,  very 
closely.    And  this  was  one  of  the  two  cases  where  the  null  hypothesis  of 
equivalent  completion  rates  between  conditions  could  not  be  rejected. 
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These  estimated  treatment  effects  represent  the  difference  In  completion 
rates  between  conditions.    Put  another  way,  the  weights  represent  the  change..  - 
in  a  student's  likelihood  of  completing  a  course  With  credit  if  he  were  to  be 
assigned  to  the  TICCIT  program  instead  of  a  lecture  class.    A  weight  of  -.359 
corresponds  to  a  decrease  of  35.9*  in  a  student's  probability  of  course  comple- 
tion. Hence,  if  50  students  out  of  an^original'  enrollment  of  100  receive  grades 
with  credit  in  lectur.e  sections,  then  "l4 -s'tudeht>  x>ut  dl  every  100  enrolled 
for  th,e  same  course;  on  the  TICCIT  system  wo*uld  be  ^pected  to' receive  credit. 
M^tlp'ly/the  regression  weights  for  instructional  condit^b  by  100  and  that 
/  usually  negative  number  jfves  , the  impact  oi  theJICCIT  program  <£n  /course 
\  CC!fS1ttion  Per  block  of *100  students  enroled  in  TICCIT  classes.     It  tel^how 
%  P^tudjnts  injone  hundred  wotfa  be  ejected  to'  receive  couVse  credi/in 
-  sections  but  fdfied  to  do  so  under "the  TICCIl' program.    The  lost  * 

£^*tj(^calcH1fted -this^ray  amount  to  a  arable  isum  across^couries  and 

clearly  negativeftreafrmeijtt  impact  shows  despite  adji 
student  trial t's\v   Entrancf  knowledg^^he  subject  matter'  is  another  factor  in 


terms. 
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*  .case  at  Phoenix  anil  fbst  -cases  At  Alexandra.''  The  ^i^te*-treatinenT<i^ects, 
'however  * .  already 
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.not  Change-  the  regression  weight  for  instructional  condition:    t  remains 


,-he  signif*<iante  of  othefr  ih^enoerit  variables  does 


\  the  same.A  Except  Where  a  regression  model  for  a  course  als>  has  a  term 

•  indicating  an  interaction  between  condition  and  another" variable ;  then  T  is  a 
S  I  ' '  '  ;■  J  *  * 

function  of  the  regijeasion  coef f icientt  for  both  the  instructional  condition 

and  the  interaction^ 

tth  007  in  the  spring  semester, 
^t  necea^rates  separate  estimates'  of  ;TICBIJ^^t\^n.^  sub^opulations. 
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For  part-time  students  the  weight  given  for  condition  holds  as  the  estimated 
treatment  effect.    But  for  full-time  students  there  is  another  term  to  consider. 
The  difference  between  conditions  is  still  the  sum  of  those  weights  involving 
the  TICCIT  program:    -.615  for  condition  and  .289  for  condition  and  student 
status.    The  latter  weight  only  applies  for  full-time  students  so  T  for  that 
subpopulation  is  -.326  while  -.615  holds  for  part-time  students.    This  would 
imply  greater  difficulty  fop  part-time  students  in  completing  a  math  course  on 
the  TICCIT  system.    However,  it  is  an  illustrative  rather  than  conclusive 
treatment-trait  interaction  since  it  does  not  occur  elsewhere. 

There  is  another  interaction  notable  for  its  absence.    If  students  re- 
enrolled  for  a  course  on  the  TICCIT  system  and  succeeded  in  fulfilling 
course  requirements  at  an  improved  rate,  there  should  be  a  significant 
term  involving  condition  and  enrollment  in  course.    No  evidence  was  found 
supporting  an  improved1  completion  rate  among  students  repeating  a  course  s 
under  the  TICCIT  program.    Even  the  regression  weights  for  student  status 
by  itself  indicate  only  a  slight  advantage  for  students  reenrolled  in  a 
course  over  those  enrolled  for  the  first  time. 

But  contrary  to  the  trend  of  negative  results  for  the  TICCIT  program, 
there  was  no  significant  difference  between  conditions  for  Alexandria's 
Algebra  I  in  the  winter  and  spring  quarters.    Yet  the  overall  completion  rate 
in  TICCIT  classes  was  .24  for  these  two  terms,  and  rates  among  lecture  sections 
would  usually  reach  about  twice  that  figure.    In  the  winter  quarter  the  rate 
for  lecture  sections  was  well  above  that  for  TICCIT  classes.    Perhaps  the 
sample  size  represents  part  of  the  explanation  for  no  significant  result. 
The  statistical  test  associated  with  the  condition  variable  for  Math  106  in 
the  fall  semester  was  significant;  this  course  had  a  much  larger  number  of 
students  and  an  even  smaller  regression  weight  for  condition  than  those  for 
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Math,  31  in  the  winter  quarter.    By  the  spring  term  at  Alexandria  a  decision 
had  been  made  to  discontinue  the  Math  31  sequence.    Perhaps  this  decision, 
along  with  a  surprising  decline  in  the  lecture  completion  rate  through  the 
academic  year  (see  Table  4.3),  further  accounts  for  Alexandria's  Algebra  I  as 
the  exception  to  our  otherwise  significant  findings,    indeed  the  lecture 
completion  rate  dropped  to  half  of  its  fall  level  by  the  spring  quarter. 

While  the  enrollment  and  completion  rate  in  lecture  sections  ffll  across 
quarters,  enrollment  increased  and  completion  held  relatively  stable  for 
Math  31  sections  under  programmed  instruction.    So  the  same  reasons  do 
not  seem  appropriate  as  explanations  for  no  significant  differences  between 
the  programmed  and  TICCIT  conditions.    For  these  contrasts  two  cases  of  the 
three  possible  tests  were  not  significant.    There  was  one  quarter  in  which  the 
completion  rate  under  programmed  instruction  surpassed  that  for  the  TICCIT 
program.     But  this  was  the  fall  quarter  when  the  computer  system  was  at  its 
most  incomplete  and  unstable  stage  in  the  demonstration. 

The  results  of  comparisons  between  programmed  and  TICCIT  instruction 
should  not  be  interpreted  as  conclusive.    There  were  only  three  test  cases 
that  involved  both  conditions.    And  there  was  a  considerable  amount  9f 
shifting  between  conditions  (see  Table  4.1).     But  the  completion  rates 
observed  in  TICCIT  classes  and  programmed  instruction  sections,  together 
with  the  historical  data  from  audio-tutorial  classes  at  Phoenix,  do  suggest 
40}     comparable  outcomes  among  self-paced  programs. 

4.4.b  English  Courses 

Much  of  the  discussion  that  preceded  our  presentation  of  the  analyses 
for  math  courses  applies  to  English  courses  as  well.    Again  our  estimates  of 
treatment  effects  depend  on  a  sample  of  students  always  less  than  total  course 
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enrollment.    This  reduced  number  of  students  can  be  attributed  to  early  course 
withdrawals  and  a  few  isolated  irregularities  in  data  collection.    Again  the 
population  that  concerns  us  is  the  total  student  body  registered  for  a  course, 
tout  interactions  between  conditions  and  other  variables  will  Require  estimates 
*>f  treatment  effects  for  subpopulations .    And  as  before  the  regression  weight 
for  condition  corresponds  to  the  estimated  treatment  effect  (t)  and  represents 
the  difference  in  completion  rates  between  classes  on  the  TICCIT  system  and 
lecture-discussion  sections.    The  analyses  for  English  courses >  however,  show  a 
significant  trait- treatment  interaction  more  often  than  not  and  in  such  cases 
the  weight  for  condition  alone  is  an  incomplete  estimate  of  TICCIT' s  Impact. 

Tables  4.8  and  4.9  present  the  regression  models  for  completion  of 
English  courses.    Three  of  the  five  course-within-term  analyses  show  a  signi- 
ficant negative  effect  for  the  TICCIT  program.    But  in  two  of  these  three 
cases  the  size  of  the  effect  is  a  function  of  a  student's  pretest  score.  There 
is  an  obvious  split  both  in  terms  of  the  significance  of  TICCIT's  impact  and 
in  whether  the  size  of  that  effect  depended  on  entrance  ability.    We  turned  to 
analyses  of  completion  as  a  program  outcome  taken  across  terms  in  the  hope  that 
a  clearer  picture  would  emerge.    Both  across  semesters  at  Phoenix  and  across 
quarters  at  Alexandria  the  value  of  the  t-statistic  associated  with  the  TICCIT 
condition  is  significant.    Now  the  trait- treatment  interaction  comes  into  the 
model  for  Phoenix  where  it  had  been  absent  and  no  longer  enters  into  the 
Alexandria  model.    There  is  apparently  no  simple  conclusion  to  be  drawn  about 
TICCIT's  impact  on  completion  in  English  coursed 

*    The  estimated  effect  (t)  for  those  cases  in  which  the  impact  was  not 
significant  still  show  a  negative  direction.    For  English  019  and  English 
029,  the  day  and  evening  versions  of  essentially  the  same  course,  the  weight 
for  the  condition  variable  in  the  regression  model  is  -.12  in  the  spring 
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ft*wr  of  Students 
Multiplt  Correlation 


Independent  Varisfelts: 

Condition: 
IICCIT(1)/L«ctorf(0) 

Intranet  ability: 
Essay  Pretttt 
Objtctiva  Protest  * 

Division: 

»sy(l)/Evtning(Q) 
It  ra: 

Spring(l)/Ftll(0) 

Instructor: 
11 
12 

13 

Student  status: 
Full-tine (l)/Ptrt-tine(0) 

Enrollntnt  in  courts: 
Rep>ti^oji  oi  course (1)/ 
Firet  enrollment (0) 

EnploTsssnt: 
Pull-tine  wrk(3)/ 
Part-tine (2) /None (1) 

Maturity: 
Age (Tears) 

Sei: 

Hele<l)/Fentle(0) 

Interaction: 
TICCIT*  Objective  Pretest 

Constant— 


*p<.05 
**p«.01 
***p<.001 

:* 

d 
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CourM  tatas.  iMlFsis  of  orrtss  with 


Credit 


English  19  and  29 
Calendar  Tear  1976 


Spring  Sewstsr 
H  -  217 
R  -  .21 

Rtgrtsslon      Value  of 
"P1*1*  t-ststlstic 


-.121 

-.016 
-.001 

-.076 


.150 

.008 

-.078 

.017 
.006 
-.047 

.608 


■1.73 

-.33 
-.13 

-.87 


1.28 

-.11 

-.60 

.37 
1.46 
-.65 


Ml  Sesastsr 

H  -  423 
1  ■  .30 

Rtgrtttlon      Value  of 
t -statistic 


-.172 

.063 
.007 

.040 


-.075 
.166 


.023 

-.357 

.004 
.004 
.004 

.316 


-3.40*** 

1.83 
1.87 

.56 


-.97 
2.22* 


.39 

-2.87** 

.13 
1.04 

.10 


Across  Ssasatsrs 
N  -  640 

OR  -  .23 

Rtgrtsslon      Vtlut  of 
w**«ht  t-etatlatii;. 

-.396 


.042 
.000 

-.013 

-,.020 

-.022 
.163 
-.0*4 

.023 

-.229 

.011 
.004 

-.001 

.011 
.510 


1.49 

-.24 

-.48 

-.34 

2.25* 

-.92 

.49 


-2.60** 

.39 
1.50 

-.02 
2.01* 


Table  4.8 
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Course  GMflitioa  btNl  Aulyiii  of  Ondei  vltk  Credit 
Aluntrli 
Wish  111 
Cileoder  Tur  1976 


fin  Mncer  Quarter              Spring  Quarter                Pill  Quarter               Acroos  Quartsrs 

Into  of  Bt*ats  I-1J8                    » -  279                    1  -  231                    II  -MS 

Multiple  Comlstlou  I-1M                    R -  .25                    I -.43,                  R-.34  1 

Regression    Vslus  of        Rs|rossion    Vilut  of        Ragntiloa    Velue  of        Regression    Vslue  of 

toteLlHkfelH'  t-ststlstlc     JflflL  tmtlitlc       Wsliht     t-ststlstlc       Wsliht  t-ststlstlc 

Condition! 

TICCITtt)/Ucturs(0)  -I.245      -3.56***           -.087      -1.04             -.689      -3.06**            -.140  -2.97** 
htrtact  ability: 

Em?  Pretest  .008        .15              .  073       1.(8              .  032       1.21               .048  1.82 

Objsctlve  Prstsst  .004        .  51              .00]        .78             -.004       -.58              .  006  2.08* 

Division: 

ty(l)/Evmlo|(0)  .456       3.33***            .063        .  72             -.086       -.97               .135  2.45* 

Clssst  j 

Liboritory(l)/Ii(ulir(0)  .271       2.49*              .047        .73                                          .108  *    2.04*  * 

Tin: 

Hintsr(l)/Fall(-1)/Spring(0)  -.091  j.49* 

Sprln|(l)/Fall(-l)/Vintsr(Q)  r                  .028  *1.02 

Instructor: 

11  '  -.011       -.10             -.152  -2.25* 

12  -.260  -1.62  -.432  -3.00**  ft 
»  '                                                                -.008       -.07              -.196  -2.10*  n 

14  '  .175       1.73               .096  .97 

15  .0!o  .81 
»  -.079  -.88 

Stwknt  status: 

FulMine(l)/Part-tine(0)  .084        .Bl              .066        .97           .  -.040       -.48               .047  1.03 

Enrollment  In  course :    •  ' 
Repetition  of  course (1)/ 

Flret  enrollment (Q)  -.026       -.21              -.253      -2.50*            -.276      -2.46*            -.189  -2.98** 


Enploynent: 
Full-tlM  w>rk(l)/ 

Pert-ti*(2)/»one(l)  .159       3.01**         '  '-.012       -.31  -.074      -1.57  .006  .24 

Heturlty: 

Age(Teere)  _  .013       2.28*              .005       1.38  .011       2.33*  .MB  3.07** 

Sex: 

Nale(l)/Fensle(0)  -.003       -.04              .009        .15  -.100      -1.72  -.030  -.83 

Interaction:  * 
TICCIT*  Objective  Pretest  .032       2.74**  .021  2.57* 

TICCII*  12  .529'      2.87**  "         .420  2.58** 

TICCIT*  16  -.413  -2.77** 


Constent-       •  -.420  .318  .805  '  .170 
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•sassier.    Judging  from  its  associated  t  test,  such  a  wight  would  be  expected 
to  occur  by  chance  alone  in  about  10X  of  all  similar  cases.    It  la  unlikely 
but  not  aignif leant  at  accepted  leveli.    For  English  111  in  the  spring  quarter 
at  Alexandria  the  estimated  treatment  effect  is  -.09.    Just  chance  occurrences. 

I  e 

instead  of  treatment  conditions  could  well  account  for  this  effect.  Bach 
of  these  cases  involves  over  200  students  so  neither  can  be  faulted  for 
small  sample  size.    Nor  is  the  estimated  effect  far  different  from  the  tabled 
coiqpletion  rates  for  these  or  other  terms  (see  Tables  4.4  and  4.5).  Quite 
simply  there  was  no  significant  difference  between  conditions  for  these 


two  cases. 


In  the  fall  semester  at  Phoenix  the  completion,  rates  were  .51  for  TICCIT 
classes  and  .69  for  lecture-discussion  sections.    The  estimated  treatment 

effect  is  a  close  match  to  their  difference:  t  is  -.172.    Here  the  difference 

"     J  # 
is  statistically  Significant  (t  -      -3.40,  £  <  .001).    And  there  seen  to  be  no 

Vasis  for  challenging  the  result.    This  case  has  the  highest  enrollment  of  any 

English  course  in  any  term  as  well  as  the  largest  nunfcer  of  students  learning 

writing  skills  on  the  TICCIT  system.     It  is  true  that  the  multiple  correlation 

is  low,   .30,  and  the  regression  model  therefore  accounts  for  a  scant  amount  of 

outcome  variance.    But  the  dependent  variable  is  just  a  dichotomous  on*  with 

values  of  1  for  receipt  of  course  credit  and  0  otherwise,  and  none  of  our 

models  attains  a  high  multiple  correlation  with  completion  (except  for  Math  32 

in  the  winter  quarter  where  the  sample  is  very  small).     Still,  the  effect  is 

greatly  reduced  from  the  size  of  those  found  in  math  courses. 

Two  analyses  for  courses  within  a  term  remain:    Alexandria's  English  111 

in  the  winter  and  falj  quarters  of  1976.    Both  involve  a  trait-treatment* 

interaction  term.    With  such  a  term  in  the  model  the  weight  for  the  condition 

variable  by  itself  gives  an  exaggerated  and  incorrect  picture  of  the  treatment 
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•ff.ct.    The  sis*  of  a  correct  eetiaate  depends  on  a  trait  aa  well.  Bar* 

f 

that  trait  la  a  etudent'a  entrance  ability.    And  becauee  antranca  ability 
ia  a  etudent'a  acora  oil  tha  objectiva  prataat  for  writing  akilla.  thejre  ia  not 
a  aingla  aatimata  that  describes  tha  treatment  affact.    Inataad  a  aariaa  of 
af facta  exiata  along  the  range  of  pretest  scores.  v 

Each  estimated  treatment  effect  ia  a  function  of  the  regression  weights 
condition  and  entrance  ability  as  well  as  actual  preteat  acora.  Judging 
by  the  sire  and  direction  of  the  condition  weights  that  appear  for  the  winter 
arid  fall  quarters  of  English  HI  (-1.245  and  -.689),  there  is  a  high  negative 
effect  associated  with  low  pretest  scores.    However,  if  we  consider  the  highest 
-    possible  pretest  score  (40)  along  with  the  Coefficients 'for  the  interaction 
terms  (.032  and  .021),  the  product  of  maximum  score  and  coefficient  exceeds  the 
8i-ze  of  the  coefficient  for  the  condition  term  alone:     t  -  -1.245  +  .032(40)  - 
.035  and  t  -  -.689  +  .021(40)  -  .151  for  the  winter  and  fall  quarters,  respec- 
tively.    Thus »  under  the  best  circumstances,  the  TICCIT  program  might  result 
in  a  slight  It  positive  effect  on  completion. 

Actually  there  is  one  estimate  that  represents  the  treatment  effect 
averaged  across  the  entire  population.    Even  for  cases  in  which  an  interaction 
term  enters  into  the  regression  model,  we  can  use  the  average  trait  value 
to  obtain  an  average  treatment  effect.    Taking  the  weight  for  condition 
plus  the  product  of  the  interaction  coefficient  and  the  mean  pretest  score 
gives  the  average  estimated  treatment  effect.   .For  the  winter  quarter  this 


would  be  -* 


t  -  -1.245  +  .032(28.71) 
and  for  the  fall  quarter   J 

x  -  -.689  +  .021(25.68). 
If  in  addition  we  estimate  the  treatment  effect  for  pretest  scores  about  two 
standard  deviations  on  either  aide  of  the  mean,  our  estimates  cover  the  range 
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of  actual  effects  well.  "  For  the  winter  term  the  average  T1CC1T  effect  is  -.33 
and  the  extreme  effects  -.76  and  .03.    For  the  fall  term  the  average  TICCIT 
effect  is  -.14  and  the  extreme  effects  -.47  and  .17.    Thert  is  evidence  that 
student  work  on  the  TICCIT  system  had  a  differential  Impact  on  course  comple- 
tion.    It  hindered  students  of  very  low  entrance  ability  and  benefited 
those  of  very  high.    And  this  evidence- .is  drawn  from  an  adjunctive  application 
of  the  TICCIT  program. 

Several  points  might  lead  us  to  question  whether  a  differential  treatment 
effect  in  fact  occurred.    There  were  nearly  a  hundred  interaction  terms 
checked  for  possible  inclusion  in  our  models  of  completion  for  English  courses. 
So  a  few  interaction  terms  could  be  expected  to  reach  a  significant  level  in  a 
statistical  test  by  chance  alone.     But  chance  would  be  strongly  against  the 
repetition  of  the  same  interaction  term  as  happened  in  the  winter  and  fall 
quarters  of  English  111.    However,  the  same  term  does  not  come  into  the 
regression  model  for  completion  across  quarters  .at  Alexandria .     Instead  we 
find  a  number  of  significant  terms  involving  instructors,  which|s^gests 
instructor  effects  as  a  plausible  alternative  to  a  differential  tSLtment 
effect.     But  the  fall  quarter  had  the  best  control  for  instructor  effects  and 
here  the  interaction  appears' in  the  model.    Perhaps  data  from  the  spring 
quarter  masked  the  dif f erentialgkf ect  in  the  regression  model  for  multiple 
terms.    Maybe  the  interaction  is  peculiar  to  Alexandria.    But  the  multiple 
term  model  for  Phoenix  also  includes  Wetest- condition  as  a  trait-treatment  - 
interaction  term.  '        ■  I 

We  tend  to  believe  that  the  TICCIT  program,  as  implemented  for  the- 

•  ••  V  » 

demonstration,  exerts  a  differential  effect  on  student  completion  of  English- 
courses.    The  effect  is  most  pronounced  for  students  least  capable  in  terms  of 
their  entrance  knowledge  of  writing  skills..    For  these      udents  the  TICCIT 
program  leads  to  fewer  completions,  anywhere  firom  about  twenty  to  over  seventy 
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fewer  completions  per  hundred  students,  than  the  lecturer discussion  completion 
rate  would  yield.     At  tfte  other  end  of  the  range  in  pretest  scores,  the  T1CC1T 
program  probably  results  in  a  completion  rate  equivalent  to,  or  higher  than, 
the  rate  for  lecture-discussion  classes.     The  data  suggest  that  there  is  a 
relationship  between  objective  pretest  and  cour^  in  TICCIT  classes 

while  none  seems  to  exist  in  lecture-discussion  sections. 

Apparently  students  who  enroll  in  a  course  again  stand  a  poorer  probabil- 
ity of  completion  than  students  enrolled  for  the  first  time.    The  statistical 
test  associated  with  enrollment  status  is  usually  significant.     In  fact,  the 
variable  representing  prior  enrollment  is  not  significant  in  only  two  cases. 
And  these  two  correspond  to  the  initial  term  of  the  demonstration  period. 
Otherwise  the  percentage  of  reenrolled  students  finishing  a  course  was  about 
twenty  points  lower  than  the  percentage  of  students  completing  a  course  in 
their  first  attempt.     Even  when  we  look  to  subsequent  terms  the  completion  rate 
for  TICCIT  classes  does  not  improve^ 

4.4.c  Estimated  Treatment  Effects 

A  summary  of  estimated  treatment  effects  (t)  appears  in  Table  4.10^ 
the  seventeen  analyses  we  conducted  for  courses,  within .  teriifep^Lrteen.    j  ..^ 
indicated  that  the  TICCIT  program  had  a  significant  negative  ^impact  on  course 
completion.    Ail  of  these  results  referred  to  outcomes  under  ^ture-discussion 
conditions  as- a  relative  standard  for  comparison  and  were  basejd  on  our  #def ini- 
tion  of  completion  as  receipt  of  course  credit  within  an  academic  term.    So  t 
each  estimated  treatment  effedt,  When  negative  in  direction,  corresponds  to  a 
loss  in  earned  credit  hours  due  to  TICCIT.    The  prfltffc-of  100  times  an  .* 

r  ' 

estimat%  effect  tells  us  the  number  of  student4#Wfr  100  enrolled  in  a 
course.iwho  did  not  complete  course  requirem^^^  the  TICCIT  system  but 


Estimated  Treatment  Effects'  on  Course  Completion 


College 


Term 


Course 


Phoenix 


Alexandria 


<.05 
<.01 
<.001 


Fall  1975 


Spring  1976 


Fall  1976 


Fall  1975 
Winter  1976 

Spring  1976 

Winter  197& 
Spring  1976 
Fall  1976 


Effect  (t)  t-Statistic 


Population 


Math  007 

-.359 

-8.76*** 

All  Students 

Math  106 

-.155 

-2.27*  - 

All  Students 

Math  108 

-.414 

-5.93*** 

All  Students 

Math  007 

-.32$ 

-10.77*** 

Full-time  Students 

Math  007 

-.615 

-6.32*** 

Part-time  Students 

Math  106 

-.470 

-7.70*** 

All  Students 

Math  108 

-.413 

-A  HQ*** 

Ail    nt  1 

All  Students 

Math  117 

-.488 

•  ~  WW 

All    n  i.  i  _  . 

All  Students 

English  19,  29 

-.121- 

-1.73 

All     O .  t 

,  All  Students 

English  19,  29 

-.172 

-3.40*** 

W  •    I  w 

'   A1  1     Cf  11/4  Ant*  a 

fiath  31 

-.446  , 

-7.07*** 

All  Students  4 

Math  31 

-.176 

-1.39 

All  Students 

Math  32 

-.472 

-4.03*** 

# 

.16 

All  Students 

Mat-h  11 

1UII.11  Jl 

A  •%  A 

.019 

All  Students 

Math  32 

-.490 

-4.01*** 

All  Students 

English  111 

-.333 

-3.50*** 

Dependent  on  Ability 

English  111 

-.087  , 

-1.04 

All  Students 

English  111 

-.139 

-2.33* 

Dependent  on  Ability 

f 
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would  be  expected  to  do  so  under  lecture  conditions*    Or  the  estimated,ef£^ct 
can  be  interpreted  as  the  change  in  a  student's  probability  of  complet^j^: a 
course  if  he  enrolls  in  a  TICCIT  class  instead  of  a  lecture-discussion  section. 

Only  two  of  twelve  analyses  for  math  courses  failed  to  demonstrate  a 
significant  TICCIT  effect.    There  is  reason  to  doubt  these  two  results  since 
sample  size  was  small  and  completion  rate  declined  across  quarters  for  lecture 
'sections  but  held  stable- and  low  for  TICCIT  classes.    At  Phoenix  where  the 
TICCIT  application  was  mainline  in  nature,  the  effect  on  completion  was  signi- 
ficant and  negative  in  all  math  courses.    The  evidence  is  conclusive  that  the 
TICCIT  program  had  a  detrimental  effect  on  student  completion  of  math  courses 
and,  hence,  on  the  productivity  of  the  community  colleges'  math  curriculum. 
However,  this  TICCIT  effect  is  not  unlike  that  found  with  other  self-paced 
instructional  programs. 

The  results  from  English  courses  do  not  suggest  as  strong  a  conclusion  as 
those  from  math.    A  significant  negative  impact  on  completion  was  indicated  in 
three  of  five  cases*    Certainly  such  an  incidence  of  statistical  tests  signifi- 
cant at  accepted  levels  of  probability  does  not  come  about  through  chance 
occurrences.     And  those  cases  where  the  effect  was  significant  include  one 
with  the  highest  enrollment  among  English  courses  and  another  with  the  most 
control  over  instructor  effects  in  the  evaluation.    Yet  the  average  effect  ^ 
size  in  English  courses  is  half  that  in  m&tlT  courses,  and  the  effects  were  less 
detrimental  for  more  capable  students  (and  even  slightly  positive  for  the  most 
capable  students) .    The  TICCIT  effect  on  completion  of  English  coursed  %fi 
neither  as  strong  nor  as  prevalent  as  the  estimated  effects  for  math  courses* 
But  there  was  a  negative  impact  associated  with  the  use  of  the  TICCIT  system, 
especially  for  students  weak  in  their  prior  knowledge  of  writing  skills* 
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4.5  Enrollment  in  Subsequent  Terms 

The  prlafp  reason  behind  TICCIT's  negative  effects  on  completion  is  • 
evident  from  the  grade  distributions  themselves.    Nearly  half  of  the  grades 
for  math  courses  on  the  TICCIT  system  was  the  one  that  signified  effort  by  the 
student  but  insufficient  achievement  /to  warrant  receipt  of  £ourse  credit  .(see 
percentage  of  Z  grades  at  Phoenix  and  R  at  Alexandria  in  Tables  4.2  and  4.3). 
Students  had  to  enroll  in  the  course  again  and  satisfy  all  course  requirements 
in  order  to  receive  credit.    But  did  these  students  reenroll  for  the  same 
course?    One  of  the  original  goals  of  the  TICCIT  program  was  to  Improve 
students'  approach  toward  the  subject  matter  as  reflected  by  increased  enroll- 
ments in  advanced  courses.     Clearly  the  completion  rates  preclude  such  a 
trend.     But  what  happened  to  the  enrollment  in  subsequent  terms  as  a  result  of 
the  TICCIT  j^ogr am?  , 

Here  we  present  descriptive  data  on  student  enrollment  in  the  term  after 
their  exposure  to  the  TICCIT  system.     Because  the  demonstration  period 
for  English  courses  spanned  a  calendar  year  interrupted  by  a  summer,  the 
data  concern  subsequent  term  enrollment  for  math  courses.    Math  courses 
involved  a  far  greater  percentage  of  grades  without  credit  and  the  fixed  • 
sequence  common  in  courses  (e.g.,  Algebra  I,  Algebra' II)  made  it  easier 
to  detect  enrollment  trends.    Also  the  demonstration  period  for  math  courses 
offered  as  part  of  the  TICCIT  program  covered  the  1975-76  academic  year,  and  so 
we  expected  a  more  natural  enrollment  pattern  from  term  to  term. 

Exactly  50%  of  the  students  who  had  received  a  grade  of  reenroll  in  a 
TICCIT  class  registered  for  the  same  course  again.    Of  the  268  students  given 
a  grade  of  Z  in  Math  007,  106  or  108  in  the  fall  semester,  134  registered  for 
the  same  course  in  the  spring  (see  Appendix  N).    The  course  repetition  rate 
among  students  given  reenroll  grades  in  lecture  sections  was  only  18%. 
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At  least  'the  studeirtts  at  Phoenix  returned  in  the  next  ^semester  and  tried  to 
fittfsfestheir  course  studies  on  the  tlCCIT,,  syatem.    For  we  know  that  there  was 
little  chahging  of  conditions  from  term  to  t*xm)     students  evidently  did  not 
•wish  to  sacrifice  what  progress  they  had  already  made  under  one  condition. 

At  Alexandria  the  percentage  of  students  repeating  a  course  out  of  those 
given  reenroll  grades  varied  little  across  conditions.     It  was  about  20% 
from  the  fall  to  winter  quarters  and  clofce.  to  50%  from  the  winter  to  spring 
quarters.    There  was  greater  variance  across  quarters  than  a$Kf>ss  TICCIT, 
programmed  and  lecture  conditions.     Perhaps  the  college's  decision  to  abandon 
the  Math  31  sequence  played  a  role  in  the  sharp  increase  for  the  winter-spring 
return  rate.    It  was  the  student's  last  chance  to  fulfill  course  requirements 
left  outstanding  from  a  prior  term.    That  there  is  do  difference  in  the  rates 
across  conditions  should  be  expected.  / 

The  rates  at  which  students  elected  to  take  a  more  advanced  math  course  in 
a  subsequent  term  were  fairly  consistent  across  courses,  terms  and  colleges. 
Approximately  50%  of  the  students  who  had  received  course  credit  in  one  term 
registered  for  a  more  advanced  math  course  in  the  next  term.    The  rates  did 
increase  from  the  fall-winter  quarters  to  the  winter-spring  quarters  at 
Alexandria,  but  probably  for  much  the  same  reason  that  the  repetition  rate  jumped 
Admittedly  differences  exist  between  Venditions  as  can  be  seen  from  the  data 
breakdown  in  Appendix  N.     Sometimes  these  favor  the  TICCIT  program  and  other 
times  traditional  practices.    However,  the  number  of  students  receiving 
credit  is  often  so  small  that  we  shbuld  collapse  across  courses  and  terms 
for  a  more  stable  picture.     It  is  then  that*  rates  for  further  study  appear 
comparable  across  conditions.    Exposure  to  the  TICCIT  program,  therefore, 
does  not  seem  to  encourage  or  discourage  further  matWfetudy  (see  also 
Chapter  7:     Student  Activities  for  supporting  data). 

see 
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Collapsing  the  counts  presented  in  Appendix  N  across  grades  gives  another 
picture  of  what  happens  in  student  enrollment  for  subsequent  terms.  The" 
percentage  of  students  returning  to  the  college,  repeating  a  math  course 
or  electing  a  more  advanced  course  appears  for  all  studen^  regardless  of 
prior  grades.    And  these  totals  better  reflect  TICClT's  overall  impact  on 
college  enrollment. 

Tables  4.11  and  4.12  present  summaries  of  student  enrollment  from  one 
term  to  the  next.     In  Table  4.11  the  last  two  rows  of  totals  by  condition  refer 
to  the  cumulative  trends  across  math  courses.     Thus  J29  of  the  435  students  who 
had  taken  a  math  course  on  the  T1CC1T  system  in  the  fall  semester  returned  to 
the  college  for  the  spring.    Of  these  returning  students,  147  were  repeating 
the  same  math  course  they  had  taken  in  the  previous  term.     Fully  34%  of  the 
students  enrolled  for  a  math  course  under  the  TICCIT  program  in  one  term 
repeated  the  same  course  again  in  the  next  term.    Granted  that  almost  all  these 
students  were  continuing  rather  than  repeating,  the  figure  to  us  is  still 
staggering.    One  of  every  three  students  who  started  a  math  course  on  TICCIT 
came  back  for  the  same  course  in  the  next  term. 

If  the  repetition  rate  for  lecture  sections  were  as  high  or  if  the  return 
rate  for  TICCIT  classes  were  high  enough  to  compensate  for  differences  in 
repetition,  either  would  temper  our  interpretation  of  TICClT's  apparent  impact 
on  course  repetition.     But  fewer  than  10%  of  the  students  from  lecture  sections 
repeated  the  same  course  in  a  second  term  and  the^dif  f  erence  in  return  rates, 
76%  for  TICCIT  and  72%  for  lecture,  hardly  accounts  for  the  gap  in  repetition. 
The  proportion  of  the  student  enrollment  from  ode  term  that  elected  to  take 
more  advanced  math  courses  in  the  next  term  also  shows  a  negative  TICCIT 
effect,  As  might  be  expected  from  its  lower  completion  rates. 
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Subsequent  Tern  Emollient:  Summary  across  Math  Courses 

Phoenix 

v       Hath  007,  106,  and  108 
Fall  to  Spring  Semesters,  1975-76 


Number  of 
Comparison  Students  Enrolled 
Group       In  Fall  Term 

Election  of  Next 

Enrolled  for     Repetition      Course  in  Sequence/   Change  to  Basic  or 
Serine  Semester    of  Course      •  Advanced  Course       Review  Course 

I     I         N     %             N     I              N  * 

•  t  ftfl.  TICCIT  *'■  215 
Math  007 

Lecture  361 

158    73  .      65    30           26    12             19  9 
241    67        22     6           69    19             23  6 

TICCIT  144 

Math  106 

^       Lecture  163 

U7    81        55    38           24    17             10  7 
120    74        18  ■  11           50    31             U  7 

Math  108  TICCIT        '  76  ' 
Lecture  84 

54    71        27    36            5     7  3,4 
76    90        12    14         ,  24    29              5  6 

m   ,     TICCIT  435 
Totals . 

Lecture  608 

329    76       147    34           55    13             32  7 
437    72     •  52     9          143    .24             39  6 

a 

( 
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Subsequent  Term  Enrollment:  ^Summary  across  Quarters 
Alexandria 
Math  31 

Fall  to  Winter,  Winter  t$  Spring  Quarters  1975-76 


Comparison 
Group 


Math  31 
Fall  to  Winter 
Terms 


TICCIT 
Programmed 
•  Lecture 


Math  31  TICCIT 
Winter  to  Spring  Programmed 
Terms 

Lecture 


TICCIT 
Totals  Programmed 
Lecture 


Number  of 
Students  Enrolled 
in  Base  term 


Enrolled  for 
Next  Term 


N 


Z 


Repetition 
of  Course 

N  I 


Election  of  Next 
Course  in  Sequence/ 
Advanced  Course 

N  I 


116 

65 

56 

19 

16 

19 

16 

44 

20 

45 

7 

16 

7 

16 

79 

51 

65 

8 

10 

16 

20 

118 

84 

71 

34 

29 

22 

19 

31 

67 

15 

33 

11 

24 

51 

31 

61 

5 

10 

10 

,  20 

234  I 

149 

64 

53 

23 

41 

18 

90 

51 

57 

22 

24 

18 

20 

130 

82 

63 

13 

10 

26 

20 

0 
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The  descriptive  data  £  rom  Alexandria  ( see  Table  4*12)  tend  to  suppor t 
the  results  from  Phoenix*    The  totals  for  Math  31  over^he  academic  year  again 
show  a  comparable  return  rate  across*  conditions*    One  course  on  the  TICCIT 
system  seems  not  to  affect  students'  pursuit  of  college  studies*    Again  the 
repetition  rate  from  TICCIT  classes  exceeds  that  from  lecture  sections*  About 
10%  of  lecture  students  repeated  a  course  in  a  second  term,  a  Tate  close  to 
that  from  Phoenix.     But  about  one-fifth  (22. 6%)  of  students  from  TICCIT  classes 
registered  for  the  same  course  in  two  successive  terms  as  compared  to  one- third 
(33. 8Z)  at  Phoenix.    Although  the  difference  between  conditions  is  narrower  at 
Alexandria,  the  TICCIT  rate  for  repeated  course  enrollment  is  still  moire  than 
twice  that  from  lecture  classes. 


course  in  one  term  reenroll  fdr  that  same  course  in  a  subsequent  term  is  a 
moot  point.     Either  estimate  would  be  an  unfavorable  program  outcome. 
Students  who  do  reenroll  in  a  course  stand  no  better  chance  for  completion  the 
second  time  around  than  students  enrolled  in  the  course  for  the  first  time. 
From  the  standpoint  of  program  productivity  or  the  perspective  of  lecture 
results,  the  TICCIT  rate  for  course  repetition  represents  a  serious  weakness. 

We  must  remember,  however,  that  this  discussion  of  subsequent  term  enroll- 
ment has  drawn'  on  descriptive  data  from  mat^  courses.     It  would7  be  inappropriate 
to  view  the  conclusions  offered  here  as  causal  inferences  or  generalizations 
applicable  to  subjects  other  than  math.     But  the  conclusions  do  rest  on  data 
gathered  during  the  demonstration  period  of  the  TICCIT  project  and  reflect  what 
happened  in  applications  of  the  TICCIT  system  for  mainline  instruction  in  math. 


Further,  the  stability  of  TICCIT  completion  rates  over  time  suggests  that  these 
results  on  course  repetition  also  hold  over  time.     In  explaining  or  arguing  the 


Whether  one- third  or  one- 
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validity  of  these  descriptive  data  on  subsequent  teym  enrollment  we  should 
turn  to  the  summary  for  Alexandria:     the  proportion  of  students  who  took  the 
.    same  course  twice  is  comparable  in  the  progr^M  instruction  and  TICCIT 
conditions. 


A. 6  Discussion 

Any  college  program  that  results  in  a  higher  percentage  of  students 
taking  a  course  a  second  time  than  completing  it  within  one  academic  teVm 
be  said  to  be  deficient  in  this  aspect  of  student  performance.     The  TICCIT 
program  had  a  completion  rate  of  16X  across  math  courses  in  the  fall  semester 
of  1975  at  Phoenix.     And  34%  of  the  students  enrolled  for  a  TICCIT  math  class  * 
in  the  fall  registered  for  the  same  course  in  the  spring  semester.  Comparable 
rates  for  lecture  sections  were  about  50*  for  course  completion  and  about  10* 
for  course  repetition.    The  results  at  Alexandria  were  less  dramatic  but, 
consistent.    The  TICCIT  program  had  a  definite  negative  effect  on  completion 
rates  in  math  courses. 
P         Completion  rates  in  English  courses  were  less  affected  by  study  on  the 
TICCIT  system.    The  difference  in  outcomes  from  TICCIT  and  lecture-discussion 
sections  was,  on  the  average,  smaller  than  that  typical  for  math  courses. 
Still,  there  was  a  negative  TICCIT  effect  since  statistical  tests  were  fignifi- 
cant  more  often  than  not.     And  the  nature^fi  that  effect  was  quite  different  in 
English,     it  appears  that  the  TICCIT  effect  on  course  completion  varied  depending 
on  a  student's  entrance  knowledge  of  writing  skills.     With  TICCIT  rather  than 
lecture-discussion  alone,  students  weakest  in  entrance  ability  stood  a  much 
poorer  probability  of  course'  completion  while  those  students  strongest  had  a 
•lightly  higher  probability  of  course  completion. 

To  what  should  we  attribute  these  results?    It  is  common  for  developers  to 
counter  negative  findings  with  qualifications  jbout  the  status  of  their,  product 
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at  the  time  of  its  evaluation.     Certainly  such  qualifications  bear  on  the 

( 

validity  of  an  evaluation.     Systeat  reliability  or  an  incomplete  curricular 

program  might  account  for  our  negative  findings.     But  the  completion  results 

were  consistent  across  academic  terms  as  the  TICCIT  computer  system  reached  a 

high  reliability  in  its  college  operations.  /Results  too  were  consistent  across 

courses,  so  f-aults  at  just  particular  -points  in  the  TICCIT  curriculum  seems  an 

t 

inadequate  explanation  for  the  negative  effects.  Quality  of  the  program  might 
be  cited  as  the  reason  underlying  our  results.     Although  we  cannot  refute  such 

"V. 

a  general  claim,  we  believe  the  reasons  lay  elsewhere. 

There  is  a  pattern  to  the  TICCIT  results  that  invites  another  explanation. 
The  negative  TICCIT  effect  on  course  completion  rates  was  most  pronounced  in 
cases  of  mainline  instruction  and  least  evident  in  cases  of  an  adjunctive 
application.    This  suggests  extent  of  exposure  to  the  TICCIT  program  as  a 
plausible  explanation  for  TICCIT  effect;:     greater  exposure  leads  to  poorer 
completion  rates.     But  the  amount  of  time  students  spent  on  the  TICCIT  system 
contradicts  such  an  e^Lanation.     Actually,  the  average  number  of  hours  per 
student  is  nearly  the  same  in  an  adjunctive    and  mainline  application  of  the 
TICCIT  system  (see  Chapter  7:     Student  Activities). 

In  a  recent  review  Robin  (1976)  noted  **4«h  withdrawal  rates  as  a  persistent 
problem  with  another  form  of  self- paced  instruction  for  college  teaching. 
Perhaps  high  withdrawal  rates  also  affected  our  findings.     But  the  percentage 
of  such  grades  was  lower  in  TICCIT  classes  than  lecture  sections  for  those 
courses  where  theT  TICCIT  program  had  the  most  negative  effects  on  completion. 
Obviously  the  cause  of  the  TICCIT  effects  was  not  attrition:     students  tended 

to  remain  in  their  classes  and  simply  failed  to  fulfill  the  requirements  for 

s 

receipt  of  a  grade  with  course  credit. 
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As  independent  Valuators  we  can  take  advantage  of  b°,th*         \^  ^*d6i^t# 


\ 


*  ;  !t  then  becomes  much  easier  to  interpret  results  in  perspact**e         ^  fort* 

Plausible  explanations.    And  in  retrospect  it  seems,  that  >h%  ^aa^^  bctiind 
>  Jr  s  * 

the  signifcicant  TICCIT  effects  on  completion  come  from  ^he      n^^t>ai  cPnceptS 

of         program.  .  .  . 

mie  TICCIT  program  led  to  completion  fates  in  math  courses  8^ificantiy      ,  / 
pwefc  than  thope  obtained  under. lecture^conditions,  but  similar  t0     "  '    \        "  * 
Reached  by  other  86 If- paced  systems.,   both1  the  audio-tutorial  ^t^/^t - 
Preceded  TICCIT  at  Phroenix  and  the  programmed  instruction  that  6o^et^Vith  ^ 
flCCIT  at  Alexandria  resulted  in  lower  completion  rates  tt*an  lect^^ 'jj&P*f* 
with  the  outcomes  of  programmed  instruction  as  a  relative  standard  f ^  gofp***'* 
8%  there  were  generally  no  significant  STJCCIT'  effects  o*  student  Gompleti°n  , 
ot:  eventt  repetition  of  a  course.    The  major  factor  behind th6  13(5 ^  *tt/c*  °*  M 
c<>ur8e  completion  is  probably  ^  generic  one:  "  Frograma"  t^at  alloV  ^ Bt*&e** 
:fco  proceed  at  his  or  her  own  pa^e  risk  loqfcing  students  unable  to  .j.      '  th^f 
instruction.  *'  A  . .  l-^^f ' 

■a  The  j3e If- paced  nature,  of  the  TICCIT  pi'pgr^does  not  fully  tot  *P? 

faults.    A  sizable  percentage,  of  students,  perhaps  a  fourth  of  tlU>a  '  ^0  atta*d 
c„Wses  but  "make  slow  progress  toward  course  cofapletion,  devote  a  s^^nti*1 
amount  of  time  and  effort'  to  their  TICQIT  gtudles.    Yet  ti»*  on  th^  ^lCQ%t  £ 
Astern  seems  unrelated  to  any  external  measurfe,  sucte'SS  pretest  or  ^0  ttesti 
and  correlates  strongly  only  With  the  tmmbey  of  TICCIT  lessons  Paa^^^  a 
8tudent  (see  Chapter  ?:     Student  Activities) .    Tilme  alone  neither  %Uarantee3 
completion  nor  compensates  fop  weaknesses  iny^mtrance  ability  #'   •  - 

,^Hfll  ptemises 
Lnitia-t  f  Qf  the 

TICCIT  project:    Do  commuti£ty  collegeTtudents  constitute  an  appr°P\iate 


at»dience  f or  the  TICCIT  pri»|am?T-  ^e  community  college,  l*keoothet  s^ct0t&  °£ 
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*^icatl^'  <J°Ubtle8§  has  use  for  ef f ect^(J^|icula  in  the  basic  skills  such 


fft  alaeb**       JSj-Ufto.    Enrollment  at  community  collages  continues  to  grow 

,*»^ft..#»'       '^j^^^Bf.tsewhere.    It  is.  an  appropriate  target  for  a  market 

•;  -.<  .^^^Pp^rs'ity .      students  at  community  colleges  Also  makes  it 

■  ^  Curriculum  to.  anticipate  and  satisfy  the  range  of  student  ; 

^1^*  ^      n**ds.    ^  differential  TICCIT  effects  on  student  completion  of 

support  to  this  view,  as  .  do  student  reactions  to  their  ^ 

f  ^  Wfl&ildtf^  V   l**8^  Chapter  6:    Student  Attitude*)  .    Perhaps  with  a  less 

*  ^^^^^S^S^*1^^         °Up  t^e'  oUtcomes  of  the  TIcciT  program  would  improve  with 

-  ^^iird'  i°  Co^pl^tion«    And  with  a  homogeneous  population,  the  TICCIT 

v  v^tograig  talce  advantage  of  management  techniques  known  to  be  successful 

"■^/^^j^/'otto^  ^otlfta  .of  ^ei£-pflLted  instruction?    For  example,  monitoring  attendance 

■v'^^*/j^i''pr<^^^  **pllcit  incentives  for  steady  course*  progress  (Morris,  Surber  & 

Bijoi^  h*lP  motivate  students  responsible  for  their;  own  learning  and 

hereby  COlllPletion  rates.  ,  .* 

fit&^9  the/Pattertx  of  tfeie  TICCIT  eff ects  lea'ds  us  to  comment  on  another 
f  he 

*  a*pect  oi         Pto8tam.    Its  developers  initially  advocated  use  of  the  TICCIT  .\ 
x        8>aAp  t°X  main^lnfe  instruc*lotu    7118  domPuter  was  to  assume  primary  responsibil- 
*ty  for  0ti*    Ut  lnatruction*with  the  taafcher  available  for  support.  Yet 
^Ompieti^  ratft8  **re  lowest  in  TICCIT* b  mainline  applications  where  Xhe 
Compute^  **8tei\dltected  student  learning,    where  the  subject  lent  itse'lf  to 
a^tive  tedChet  p*rticipation  in* the. Instructional  process  <i;e.,  r^viiewing 
student  ^d  discussing  themes),  the  gap  in  completion  rates  between  / 

^XCCIT  Cture  conditions  closed •    Where  instructors  assiiW  primary  responsi- 

bility ^eaClllt*8  and  chose  to  use  the  TICCIT  system  as  an  adjunctive  resource, 
*he  mi*****  *IC^1  effect  *as  below  that  for  mainline  applications  and  even  \ 
^carne  ?°  •  °r  a  select  group  of  stents .  ■  > 

ERIC  \  .    .  _      ,  °\  _  < 
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Supporting  active  teacher  participation  in  applications  of  instructional 


technology  is  perhaps  too  fdmirliar  and  worn  by  frequent  use.    But  the  results 
of  the  evaluation  of  course  completion  as  a  program  outcome  would  lead  us  to 
prefe*  a  class  ^n  which  teacher  and  computer  share  the  burden  of  student 
instruction  over  one  in  which  a  computer  syg^m  directs  student  learning.  As 
we  perfect  our  understanding  of  what  facilitates  learning  and  how  technology 
be s tr con tributes  to  this  process,  it  is/'best  to  draw  on  the  experience  and 
knowledge  of  teachers. 

Our  data  show  that  the  mainline  application  of  the  TICCIT  program  had  a 
f  neg^tivQi  impact  on  course  completion  rates  at  community  colleger .  %rther, 
there* was  evidence  of  a  differential  TICCIT  effect  on  completion  rates  in  * 
English  courses.    Such  findings  suggest  the  possibility  of  ;p  systematic  . 

influence ijjfx  other  comparisons  of,  outcomes  dependent  on  course  completion.  The 

P    '  '    1  ' 

evaluaCWKof  student  achievement  may  be  particularly  susceptible  to  this 
threat ^to  validity.  The  next  chapter  in  our  report  addresses  this  matter 
further. 


J  K 
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Chapter  5 
Student  Achievement 


How  does  student  achievement  under  the  TlCcIJ  program  compare/with  that 
of  students  tatight  by  conventional  practices?    That  is  the  has  fir  question 
addressed  in  this  chapter.    It  is  not  so  straightforward  a  question' that  thtf 
process  for  finding  an  answer  is  immediately  evident-.    The  question  itself  „" 

Bf "  * 

does  imply  comparing  achievement  results  from  classes  on"  tluv**W^ 
and  results  already  attainable  through  existing  prA^Hcej^  wvTt  aim^Km 

...  -    ^-  A^i-^W 

unbiased  measures  of  achievement  for  gathering,  tlje  rel^wntydatfa;  then 
analyses  must  compensate  for  nonpquivalent  compaTis<Ji\.'grpupi,  especial 
since  thejre  were  pronounced  treatment  effects  on  comnlAtibn, ^^u^ing 
justments  for  entrance  ability  and  other  student Achaxa^te^istic's.  Th 
tests  prepared  for  this  evaluation  $nd  the  approach .'.  adopted  £or  data  "anVly^W 
(see  Chapter  2:    Methods  and  Measurement) 


)->  we  can  fcstitiat 


t  t^e 


effects  on  student  achievement  wkr^  within  the  conte^t>«f^his  demolatj 

This  chapter  portrays  the  achievement  ^outcomes  of  tjfc».T3£jfrrc  pro 

relation  to  those  of  usual"  classroom  teachings methods . 

*        /  ,  „ 

achievem^t  as  a-function  of  unique  TICCIT  attributes,  su^r  aajtfae  *PSfl 

>  "*  V  ..  o     « k  F% 

the  computer  system,  appear  elsewhere  (s£e  Chapter  7:     Student  Activit 
The  first  section  "here;  deals  with  the  measures  that  define  ^hi^Vemenb^oki'* 


our  evaluation  and  the  students'  within  the  groups  on  which  ou'r  Xompar^fi 
of  program  results  depend.    The  chapter's  second  section  presents  dUffi£&ies 


section 

of  achievement  data  similar  to  thos^lgfl ready  given  for  course  completioft;  it 
gives  descriptive  statistics  on  pretest  and  posttest  scores  by  instructional 
condition^  or  each  course.     In  the  next  section  we  turn  to  the  analy*^  and 
estimate  the  actual  TICCIT  effects  on  achievement.     Tfcese  analyses  pertain 


/ 


2?8 


to  student  performance  on  the  overall  posttests  for  math  apd  English  courses 

^  *■  \       v.;       '      ^  i 

as  well  as  on  special  tests  designed  to  assess  other  aspects  of  achievement. 
The  fourth  *nd  final  section  of  this  chapter  is  a  discussion  of  the  findings 
and  their  implications.  _     *  x 

5.1    Achievement  and  Students 

this  evaluation  treats  achievement  as  one  aspect  of  student  performance,  ♦ 
albeit  the  most  important  one  foreseen  in  our  original  plans*    Because  a 
program's  impact  on  achievement  often  determines  its  success  in  schools,  the 
tests  that  operationally  define  achievement  should  be  fair  and  comprehensive* 
*  The  measures  prepared  fo^^aluating  the  TICCIT  program  seem  to  exhibit  satis- 
factory validity  and  reliability  (see  Chapter  2:  '■■fiode  Ind  Measurement). 
This  is  a  necessary  condition  for  making  comparisons  of  program  jfesulU^but 
it  J*  not  in  itself  sufficient  for  understanding  our.  estimated  TICfclT  effec 


on  achievement, 
f  4  Our\Qi*tat 


es  of  TlCCIT's  impact  on  achievement  obviously  depeftd  on  the 


7 


measures  of  achievement 


the  students  who  took  those  tests* 


> tb^i V^n  ap  mea 

:e  between  TICCIT  classes  an0  lecture-discussion  sections  in  their 
achievement  faults  will  appear  as  a  number  of  points  on  a  test,    ^iich  a  {^oint 

difference  between  program  flkitcomes^may  seem  to  be  an  abstract  form  for 

.  ♦  /  .  * 

reporting  results.    So  we  also  tried  to  isolate  those  effects  reflected  in 

/  T 

postteats  and  checked  for  effects  on  learning  at  other  .points  in  a  course. 
Still,  every  TICCIT  effect  on  achievement  might  be  suspect.    Opily  a  select 
group  of^J^tients  completed  math  courses  on  the  TICCIT  system  and. there  was 
evidence  of  a  trait-treatment  interaction  for  student  completion  of  English 
courses  (see  Chapter  A:    Course  Completion  Rates). 
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5.1.a   Measures  of  Achij 

Each  estimated  effect  reflects  the  relative  gain  (or  l°aS*  ^chieve" 
ment  for  a  student  who  completed  a  course  under  the  TICCIla  Ijrog**10  ^Bt:**d  of 
in  a  regular  lecture-diHussion  section.    Primarily  these  esti**tC8  t0    -  • 

achievement  as  measured  byr  our  ppsttests.    An  effect  of  about  f o*r>  P°***t^  ^ 
example,  would  represent  a  difference  in  program  outcomes  of  ne*?iy  ketcf** 
in  teMMK of  total  possible  score  cm 'a  posttest.    The  effects  rep°rted 
♦  idtiievement  will  be  much  more  subtle  than  those  for  completion'-    T  ey  **Hthe* 
approach  the  size" nor  carry  the  direct  implications  of  treatment  c  .ec**  ^ 
course  completion.    In3te?d  they  represent  a  point  difference  °n  J^*^'****. 
Although**  treatment  effect  does  not  necessarily  eorresp 

and  t°>  Patttculat 


posttest  items,  it>  might  be  attributed  to  certain  types  of  items  *nd  tlx^'  ro^i 


uspLth  nfcce  specific  informa^tcta  about  the  relative  strengths  a°d  W6alt^88^ 

the  TtCCIT  program.5    For  math  courses  we  will  examine  student  achieve**t^  _t 

-.i  ,  r       "  .  « ■■'  ,         .  ,    •     .,      ■  w>  i 

of  subscores  for  ability  levels  and  -content  categories  as  well  **  J^tal  ^oatt^** 

scores  on  all  common  ol&eetlves  (see  Figute  2.3).    For  example*  *      CO**$  fot 

*  *  <&f&\  '    r  nts  mi 

solving  routine  fjroblems  *and  demonstrating  cbmprehension  of  conceP      ^fcj^  rev^dl 

TICCIT's  merits 'fo|  teaching  a  certain  typ^  of  cognitive  activity*  other* 

for  arithmetic  ogera^ns  and  linear  equations  might  define  the  pr°8ra**% 

strongest  points  along  a  dimension  representing  the  content  do**iD#  W****ln 

English  courses  subscores  along  dimensions  for  ability  and  content  ^^^^on-* 

tribute  4ess  to  understanding  TICCIT's  impact  than  would  direct  tests  0f  - 


writing  skills  and  actual  writtei^work.    We  wiiX  report  achievement  dlf**teice0 

in  English  courses  in  relation  to  both  our  objective  test  of  writing  & 

,  ■  •  ^  -P 

essay  test.    Such  detailed  analyses  of  program  outcomes  shouliPer,f^t  **Qt  00].y 
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•ummative  estimates  of  TICCiT'f  effects  on  achievement  but  also,  specific  and 

f 

con8tructive  evaiuative  findings  for  stgpngthening  the  results  of  subsequent 
T*CCIT  application. 

■  m 

Further  implications  for  instructional  applications  of  the  TlCCIT  program 
^Sht  be  derived  f  rom  measures  of  student  achievement  %£her*jkhan  those  taken  \ 
at  the  end  of  a  course*    Posttests  or  final  examinations  reflS^t  cumulative 
*e*tning  over  an  entire  course  of  study.    It  may  be  the  vcase  that  results  in 
te*tQs  of  the  ^^ulsition  or  retention  o£  material , '  taken  immediately  upon  com-.  ^ 
pleting  a  unit  o£  study  or  when  stafctifcqj  a  later  course  in  a  sequence,  differ  ^ 
to  thdse  obtained  by  posttestlng  alone*    Given  the  moduJUtfr  dfsign  of  the 


J^SCIX  curriculum  for'precalculus  mathematics,  „ students  might  acquire  the 
r^Si  aid '  concepts  f#r  a  unit,  of  material  and  forget  seme  of  that  learning  1>y 
tl*fc,  encfW  ^coHt8e.    The^  topical  testing  for  immediate  learning  would  reveal 
a  different  patten  of  effects  than  poettesting  for  ^cumulative  learning.    Or  the 
°PPortunity  for  self-Paced  learning  and  repetition  on  the  TICCIT  system  mighty 
lead  to  overpractice^and,  hence,  improved  retention  of  the  mater ill  covered  in 
a  coursed     If  that  material  serves  as  a  prerequisite  to  more  advanced^  courses, 
testing  f°r  retfctxtion  at  the  beginning  of  a  l£H*X  academic  term  would  provide 
evidence  of       achievement  effect  apart  from  that  represented  by  poettesting. 

The  measures  that  define  achievement  for  our  evaluation  should  be  sufficient 
fot  detecting  overall  effect^,  isolating  the  nature  of  these  effects,  and 
checking  £°r  *ffects  at  differed  stages  Of  leatning.    AJ1  of  these  effects  will 
b^  reported  as  point  differences  on  special  tests  prepared  for  tjie  evaluation, 
Each  point  differenCe  represents  a  real  difference  in  student  performance.  It 
ma7  be  helpful  to  consider  other  facts  in  interpreting  these  effects:  test 
^atacferistlcs  and  sample  items  (Chapter  2)f  teat  coverage  (Appendices  C,  D  , 


and  B);  maximum  j>ps  sib  le  scores  and  average  observed  scores  (Appendices  0  and 
P);  and  cerjtainly  the  context  of  the  demonstration  (Chapter  3).    But  perhaps* 
the  most  critical  factor  to  consid*  i8  the  population  of  students  who  took  a 


particular  "test. 


5«l.b    Populations  of  ^Hgnta 

One  of  the  key  as8umption|^made  in  our  data  Analyses  is  that  the^ 
is  a  population  from  w^ich  the  students  may  be  regajgp  as  representative. 
Those  students  Who"  took  posttestJs *o  not  represent  all  students  at  the  cbm- 
W^tyjco lieges  or  even  students  enrolled  for  algebra  an^composition  courses. 
Instead  >  they  were  theJp^ulation  of  -"aitudfents  who  both,,  enrolled  for  and  gP»ive| 
a  passing  grade  U§m  target -course.    Had  there  been  no  treatment  effects  on 
course  completion,  the  students  completing ^*o\irse  under  the  TICCIT program 
and  those  completing  the  same  course ^.Xecture^d^tussion  classes  would  be 
fregar^e4  *B  representative  of  tH^  «*me  population.    However,  the  TlC^I^'f^ogram 


did  affect  course  completion  rates.  jsp- 

■7. ;v*    ^urse  c<^pl«tion  rates  were  generally  lower  In  TICCIT  dlasses  tffem  in  m 

"m        '  X  V'-  *"  ?  ■«*  '^P" 

lecture-d^ttssioii  sections:    f«jer  students  rjeceived^a  passing  grade  for  a 

course  on  ^^|CICCIJ  system  |han  «o^e  exoected  if  the  same  students  had 
'tfaVen  fche  course  under  lectu^.^^^   |f  conditions*,  The  population- of  * 
Students^osttested  in  the  nctitCctMfPttWn,  tfeera^^as.  not'  equivalent  tp  " 
that  posttested  in  the  lecture  condition.    Althougk'similar  student  populations 
had  started  courses  under  both  conditions,  a  smaller  percentage  of  students 
actually  completed  courses  under;  the  TICOIT  program. 

Xt  la  irtpor^ant  to  reco^ze  exactly  the  intuitive  approach  behind  our  data 
analyses.    The  treatment^  effect8  0n  ach^^Sfcent  represent  the  point  difference 
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#jjfcfeen  the  actual  postteW  score  £o^  a  student  exposed  to  Tl^^^  an  e* ^ 
mated  postt^st  score  for  that  same  student  had  he  <?r  *he  *ns*H<i  teen  *n  5 
section.    The  reverie  of  this  procedure  would  be  |g|^t*mat*  ^  «c0* 

would  be  under  the  T ICC IT  program  foj^each  student  compl**1"*  ^  leCture  d 
But  such  a  reversal  would  be  inappropriate:    nj>t  all  st  ^^Jeti^g 
classes  would  be  expected  to  complete  the  same  course  under  th*  ^lCclT  p*°* 
The  treatment  effects  reported  for  posttest  achieveioent  rely  0n  actU*rfb** 
scores  under  the  TICCIT  program..    There  would  b#||ttlC  v***  e«^*ti^ 
effects  fcAtudents  had  tkfey  been  able  to  eon^l«*a  c°ur8e  vh*^tfrere  **** 
no  representatives  of  this  population,  y  ^  ^  .  ^  A 

This  leaves  us  with  achievement  re8uiWapplicablC  °n  y  ^  tb«  stude* 
population  capable  of  completing  courses  on  the  TICCIT  *yst«^e  j^^ngl*1* 
courses  such  a  population  has  considerable  overlap  8  CO|**ltefPart  U<* 

lecture-discuafc^on  conditions.    The  TICCIT/  ef3ffta  on  student:  ^pie£i°n ^ 
'English- courses  were  not  so  pronounced  as  to  preclude  .coftarla^  ^  achi^ 
ment.    The  effects  on  completion  did  vary  according  to  ^*ior  ^Qvj.edge  °*  "H 

writing  ^kills.    But  this  trait-treatment  interaction  seem*  t0  ^ yfctfpt** 

to  th 

fact:     our  estimates  for  achievement  effects  apply  only    °  6tudent0  ^ 

completed  English  courses  with  TICCIT,  "and* similar^  students -ut^  c^pi«fe'  *  8l* 

was  d  ^^^t 

courses  under  conventional  classroom  conditions.    Thcre        f  tl^,  a  *oUlJ  8 

for  comparing  program  results.  *  v 

The  situation  for  stiSent  achievement  in  math  couraes^Vt^  tbere  ^ 
pronounced  treatment  effects' on  course  completion*  is  diffex^t%    ffce  e9^^       1  ^ 

effects  on  math  achievement  refer  to  about  one-fifth  of  ****  *PoPU;ut*on  ^f* 

"  N  *  ^£t^ 

.  might  enroll  for  precalculus  courses.    That  cotreap<>nd8  to  ****  ^^letio* 

j>ly 

among  'students  starting  a  math  course  under  the  TICCIT  progra^     Quite  si  * 

c  ',  ■  283 


th.  oat"*"  •*  tK  fiCCtt  >>">»*«>  Mth  nf0ti  t0  cour..  co^tio0  pr.««.t 
'.^.r.!'""0-  bw         "Wfd  ^ul.tlo„  for  oth.t  of  ,tui«t 

p.rfor-r.0.-    »«t  wlot  «ri**,for  o»,  lMttual»t.  »»-  W-c^  ^  gi»«- 

end  of  «»  *cad"Wc  tJ?l°*"dU"  ■ 


/ 


<*  their  -«*«**»*^V..^f*»*'*14*' 
of  .tud^' ."^^  tied1  cl«.e.  thtougS  the  or.d  of  .  ^  ^  ^  yrgor 

trolly 


cbui 


earlier  *°  a  c°Ut 
latin*  °*  «t 
Thi#  has  be 


o£  the  original 


-ting  I**"  CU--i  lt  is  -bout/-e^t^h8 

Also.  toP^  t«.^g  forWiate|^ook  pl*J 
*S  than  P**^**,  a^2  ifc.e:;rei«lt.  b^  a  trader 
population  of  .t^^       4-  y  . 

achieved  ^Ul^    gu^e  ^^^3  -hflt  defiI1e  *chle^eut^hopld  t>eW 
in  aind  WbiXe  lut,tpreti^  effects.    ^  ^  papaX*'*^  ^  ^  acH^fe 
neot  result-  may  ^  ^rded  as  fpr.^^  certain^iUat  atJy 
inf  ereacea  »ade  ^  ^  data. 


5.2  ijesepMtiSj 


Hatisti'9         Pret-st8  aQd  pastteVts  vithtc 
in  Appeodi*  0.    ^  ^t^f  report  ^  0f  pb8ervat 


peacripttve 
[Ddi*  °« 
,o  -core 


°rt  the  number  °^b8er>attoiuj  per  c^' 
^tding  *  the8e  °b^rVatloaa,  and  a  teet»s  deV*atlon 
witbi*  each  claaa>    ^e ,  al80  pr^nt  ^  for  «U  ^ 

academic  t*  ■»«  bvfen^  _.^tt. 


l*  ^rVation^»    ■  ~  atandato  " 

*  <=o„„e  to«*l.  «*  «U  .^J^i.  » 

^•T^'^Ki.-  II0C-1'  pro.r*f4>- 
|<ve«o°r.  .to..        •  cMr.e  •irtl°  •  t,^  „o*o» 


doa^  by  condlti^p^  ea8l^  to:8ta8p  flnd  portray,  a*j  ^  the  fom, 

followed  *»>»r  ^^see  ^abl^  5.AX*    SttH,  there  ^  evidence 

in  the  feum^^a  bv     JlviduaX  ^a^^J^f^  tboae  fo^^^  coadition, 


*PM»<U„  0  1.  the  ^ce  ofSrete>'  or  Po,^  J.C    Th«*.  ^ 

,u<:h  ^  typ«'liy  '"""^M  „.«-«ii.  I..'""0*'  wii<"  **  th* 

*V,lU»H„,  or^1'"  '8<S-  „«.bers  «*«*•«*  to  »»  0kJ«tttv<i 

««  ^itloe  KSe  abruption,  to  oth,^^  ^th^ed,,^ 

Cation  «o'e  ««•  l0f  teq„e„t  durio.  *.  d^^fcO  <>«*°<i  o,  J. 
»°J«tt  „od  seldo.  ^.colLctioO  ««W.  ^  tlCClI  oond,ttoi>>  ^ 

"*»  •V..  .tho»f  tlccIT  C1....0  »«  Al8<  f  „t  ^^'"'"•  V.ct. 


P0,t^S*.  ~  •et"rtl1'  K  of  the«  studeot.  tbe 

«  ,i££lculty  oooo'ted,  tvt.  «.«  B-fM'1""  d,ti  ^.  otl-ot  Uctut;_ 
"'^too  'I""'-    "  U  dl8'Ut,tlo>,8  10  coll*"1"*  , 

t0°  ^oent  to  ^  ihreot  to  *  „W  0f  t*e 

■>t0"»t»4  «  to  .bd»d<m  f"*v  4Miyse..'  tet„  h»" 00ly  T**<*j>  : 

to  «*>  "j^eVfor  .'■•:iK  °»  the  fu*»<ntal'ie  Vrittoo 

En*Hsv    stocMts  »  <t  i6*tuieldl8c«»io°  "'t^  „et.  '"•"^te  time 

£°r  ^^etloo  the  °bJ«<=ttVe  ^  ,.„y  p0St«ste  ^  m  al„  poorly  „„  ^ 
""^e^  ■"*8°re''         '»•  1?8  of  Ajpeodlooo)-    Th>^her?. »°  reWe 
"•"V,  fot  *>  to        1»  ^Uog  falr  ccpot"""  of  acbiev^ot  r-yts  ^io 
**  8X„r  t— •    Pa"  ft°»  otWr  tor.,  wool"  b,  U,.^,^  ^ 
U  1  Xcl»l  «—         lt»  ^  idloSy4cr^»-  '"ov^^  ft  »oteJ  tb.t 

Htfor-»M  *  etUde,,l»  lW#^t  olaoo"  00  the  mccW  V«,  C0MlBteo 

'*«'  t«8ults  fro-  "thet>^  l^||l^«tlV'   U8t  Of  Tltf*  "kUl>-  »«tet 

than  *Hul»  £I°°  °thet  te^  00  the  t"t,"""l  «rt"*»W  "U  8b"ve  ■ 

""^^oce  of  tb»t  -^.%         ^iscu-Bio"  ^  ^  ^ 


:  :,v..  .«      ■        •  *  X 


li.';:V^r;^  C°^tn.  »ath  coutM.*  o££.r«I  after  A  initial  academic  t.r. 

°*  tb>  d&*0OB         ^P««*od.    Oft*n  student*  failed  to  complete  course  require- 
*  **nts  wlthio  one  acadV  and  enrolled  for~tW  same  cour..  ageing  the 

<**t  t.r».    T^,e  8t^nt8  typically  had  taken  their  cour.e'f  pr.te.t  once  and 
^nded  to, avoid  «blng  8q  ^  t^  ^  students  had  a  pretest  from 

°ne  aCadeoic  term  and  a  po8tte8t  from  another,  later  term.    It  was  important 

tK  retain  these  a**.  * 

*at  we  '  at«  since  compietlon  rates  severely  limited  the  number  . 

Of        .  ^flr^po8 1 test  oho 

preteB    v  D8ervatlo.n8.    Therefore,  whenever  a  student  had  a  pretest 

*nq  p0S^8t  t8ken  ln  8UCce88lVe  terms  an^remained  in  the  same  instructional, 
c°naltion  *°r  both  terma,  that  earlier  ^B%est  stood  as  our  measure  of  entrance 
ah-ti     .      There  were  51  . 
#•  °Uity-  1  8uch  cases  in  the  evaluation  (as  documented  in  footnotes 

^thm  Appa^lx  °>-  %  .  ' 

finally.  P"8entin8  achievement  results  at  the  course  level  omits  much  of 
^  tone  of  the    ata  evlqent  affl0ng  ^,8e8.    Although  the  average  pretest  score'  ' 
W  airoilar  across  co*^^  &  £ulJ  8tandard  deviati-|  frequently  separated 

t,e  highest  '4lass,averag,  ^  the  1    ^  q£  thc  8a»e^^.  S^fA,      .  * 
^re.5?r  ln  the  averse  postteat  score  among  classes  of  the  same 

Edition-    00  both  the  Prete8t  and  po8t test,  the  scores  of  students  completing 
*  c0ur9e  with  XICCIT  u8UaU^fonned  the%ghe8t  averaga  for  a  \  cla88/f^^ 

Sr  in  several  ase8>  TlCcix  ^a8,e8  had  ^  ^  the  highe^iean  score  Jl^^  i 
Por.  Knt* also  the  1  ,  I       '-  ':'-}''>■  ■■  v|;-'^-; *'«**,.  . 

Attest  bu  least  variance  ln  8Core8  within  a  class.    Such  consul 

h*%h  performance  might  be  attrlbuted  ^  ^  ^  ^  ^  ^ 

•VV     -   f^tricf i°n'  in  C  e  ta,n8e  of  8COres  due  to  completion  rates 

»  .,  c  *^ 

'^WP^^^li  n  feath  Courap 

^W'W^^UMBf?'*    —  ^SS  \ 

.  >        y*r     ,  *  >v  Tables  5.1  a  -     ,         „   %  .--^5? 

'sl^V!i  present  the.  achievemedt  results  for.  math  courses 

^     .lUe^  ^^le  obvious  ftOBi  >  '  ,  -«f  '  . 

ro*  their  respective  numbers  of  observe t ions,  all  pretests 


1 


/ 


>*^*pase  a  larger  student' 


encotpasa  a  larger  student  papulation  than  all  postteeta  do,    The  deacrlptiva 
statlstlca  for  air  pretfctp  repraaent  tha"  entrance  ability  of  students  starting 
parallel  data  foi/all  posttests  reflect. the  final  achievement  . 


of  just  those  atodente  completing  a  course,    ly  repeating  the  descriptive  test 
data  for  fctudentfcv  with  bath  a  pretest  and  poattest,  the  tables  give  pretest 
means  and  standard  deviations  for  just  those  students  completing  a  coflrse. 
Thus,  fyretest-posttest  observations  refer  to  a  student  population  different 
from  the  one  which  began,  a  course  but  essentially  the  same  as  that  which 
completed  a  course:     the  test  data  tiUfnge  for  the  jjtetest  but  stay  the  same 
tor  the  poattest.*    For  txpapl^  students  in  Math  007  for  the  fall  semester  of 
1975  had  a  mean  pretest  scofre  of  37.26  and  a  mean  posttedt  score  of  39.42. 
Among  students  with  both  a  pretest  and' a  posttest,  the  respective  means  were 
41.80  and  39.32,  essentially  unchanged  on  the  posttest  but  muqh"tf±«her  on  the 
pretest.  ^ 

The  average  pretest  score  for  students  completing  a  coura*  was  nearly 
always  higher  t{ian  that  for  all  studetfcs  starting  a  cdurse.    This  fact  i$  * 
especially  apparent  within  the  T1CC1T  condition.     Students  completing  a  math, 
course  under  the  TICC1T  program  often  had  a  pretest ,mean  approximately  one 
standard  deviation  higher  than  that  of  all  students  who  originany  atarted  the 
course.    Fqr  lecture  ^ections  the  corresponding  increase  was  seldom  one- third 
as  large.  •Those  students  who  completed  a.  course  on  the  TIGCit  system  certainly 
represent  a  more  select  potation  than  their  counterparts  from  iect^e  se^Jpns 
Our  achievement  analyses  deal  only  with  pretest-posttest  observations  and  yield 
estimates  of  treatment  effects  applicable  only  to  that  population  of  students 
who  completed  math  courses  under  the  T1CC1T  program. 


Comparison 
Term  _il2UL_ 


Achievement  Swary  - 
Phoentx  A. 
Math  007,  106,  108  and  llf 
.    Academic  Year  1975-76 


All  Posttests 
N\  Mean  sd 


All  Pretests 
N   Mean  sd 


9k£ 

ERIC 


Fall 

Total 

415 

37.26 

1M 

182 

39.42 

10.1  , 

TICCIT 

161 

.35.32 

10.2 

27 

44.26 

6,7 

TICCIT  (day) 

122  .  35.54  1Q.2 

19  43.21 

7.5 

TICCIT  (eve.) 

39 

'34.62 

10.1 

8 

46,75 

3.6 

Lecture 

.254 

38.49 

11.9 

155 

38. 5B 

10.4 

lecture  (day) 

164 

36.26 

11.4 

111 

39.59 

10*5 

Lecture  (eve.) 

90 

42.5.4 

11.6 

44 

36.02 

1  Vfc 

9.7 

■  Day 

286 

35.95 

11.0  . 

130 

40.12 

10:2 

Evening 

129 

40.15 

11.7 

52 

37.67 

J 1  i  V  1 

9.8 

Spring 

Total 

332 

36.24  11.3 

181 

38.96 

8.3 

TICCIT 

110  36.1$  11.4 

•35 

•  40.57 

6.9 

TICCIT  (day) 

83 

36.23 

10.8 

30 

40  03 

6.9 

TICCIT  (eve.) 

27 

35.93  .13.4 

5 

43  80 

7.3 

Lecture 

222 

36.28 

11.2 

146 

38.58 

8.5  '• 

Lecture  (day)' 

134 

35.97 

10.9 

87 

Ml 

39  37 

7.9 

Lecture  (eve.) 

88 

36.76 

11.6 

'  59 

37.41 

9.3 

Day 

217 

36.07 

10.9 

117 

39.54 

7.6 

Evening 

115 

36.57 

12.0 

64 

37.90 

9.2  1 

Fall 

Total 

178 

46.88 

11.3 

101 

32.38 

7.3 

TICCIT  ' 

98 

47.14 

11.7 

34 

36.41 

4.6 

Lecture 

80 

46.56 

10.8 

67 

30.33 

7.6 

Spring 

Total- 

229 

13.61 

3.4 

106 

32.64 

8.5 

TICCIT 

80 

13.20 

3.4 

17 

35.29 

6.5 

Lecture 

149 

13.83 

3.4 

91 

32,14 

8.7 

Fall 

Total  , 

,133 

41.68 

11.3 

54  *7.43^ 

8.6 

** 

TICCIT 

61 

44.72  Ul.l 

10  40.50.. 

'7.6 

Lecture 

72 

41.68 

11.3 

44 

36.73 

8.7 

Spring 

Total 

153' 

42.52 

10.9 

55 

39.55 

8.2 

TICCIT  ■ 

52 

43.60 

9.8 

12 

39.25 

8.3 

'  Lecture 

101 

41.97 

11.3 

43 

39.63 

8.3 

Spring 

Total 

150 

12.39 

3.7 

71 

24.58 

8.8. 

w  r 

TICCIT 

43 

13.02 

3.5 

14 

24.71 

7.7 

Lecture 

107 

12.13 

3.8 

57 

24.54 

9.2 

/• 


Pretest-Posttest  Observations 
Pretest  Posttest 
Mean  '  sd      N   Mean  sd 


152  41.80  .10.8 

22  44.86  7.8 

17  43;88  7.5 

5  48,20  8.9 

130  41.28  .11.2 

91  40.10  J0.9 

•  39  44,45.11,3 

108  48.20  7.9 

44  44,52  11.1 


172  49.05, 

32  40.06 

28  39.21 

4  46.00 

140  38.82 

83  38.19 

57  39.74 

111  38.45 

61  40.15 


10.7 
10.5- 
9.8 
16.4 
10.6 
10.1 
11.5 , 
10.0/ 
11.8 

/ 


71  53.89  i.4 

29  57.00  6.4 

42  51.74  9.0 

.91  14.59  3.2 

4  5  IMp  .3.4 

86/10*9  3.2 

47  46.60  9.9 

7  51.86  8.5- 

40/  45.68  10.0^ 


52  45.15 
•12  46.50 
40  44.75 


9i6 
9.9 
9.5 


64  13.25  3.6 
12  14.33  1.9 
52  13.00  3.9 


152 
22 


39.32 
43.32 
17  ,42.00 
'..  5  47.80 
130,  38.64 
39.96 
35.56 
40.28 


91 
39 
108 


44  /  36.96 


172 
32 
28 
,4 

140 
83 

'57 

111 
61 


38.76 
40.09 
39.64 
43.25 
38.49 
39.29 
37.32 
39.38 
37.70 


y64 

12 
52 


9.8 
6.6 
7.0 
1.9. 

10.2 
9r.9 

10.0" 
9.5 

10.2 

8.4 
7.1 
7.1 
8.3 
8.6 
8.0 
9.4 
7.7 
9.4 


71  33.86  5.8 

29  45.97  4, 

42  32.40  6.3 

91  32.09  8.7 

5  36.60  5.7 

86  31.83  8.8 


47  37.60  -8.9 

7  42.29  6.8 

40  36.78.  9.0 

52  38.96  7.9 

12  38.42  7.3 

40^39.^2  8.1 


25.34  8.6 
25.58  7.1 
25.29  9.0 


Table  5.1 


'  Achievement  Sunnier  • 
Alexandria  1 

.  Mat**,  31  and  32, 
AcadaoleiYear  1975-76 


Course 


'*ath 


.Term 
Fall 


J ,    \  ' ■  V ■  )>  All"  Pretest 
Comparison  y  .?:•'».•»' 

Group  **'  /     *  v'N  •  Mean  sd 

Tptal     ;T.  V*  ',195  31.72.  10.5 

TICCIT  t  'ytfi  90  ,31.56  11.3 
Programmed  f,»  ?   -  47,  '29.74  8f5 

Lecture     .  r   A  58  33.59  10.3" 


Winter  Total 
TICCIT 
Programmed 
'Lecture 

Spring  Total 


) 


TICCIT ,  ' 

Programmed 

Lecture 

Winter 

Tottl^ 

.  TICCIT 

Lecture 

Spring 

Tqital 

TICCIT  , 

Lecture 

1 

>■): 

J4 


105  30.00  12.0 

41  27.29  10. a 

33  32.21  12.6 

j  31  31.23  11.8 

0  'i 

130*  -29. 01."  12.0 

41  39.12  11.9 

58  26.-14  10.4 


AH  Poattesta 
•.  N  i  Meta  "  /ad 

5&j>  38.90  .6,1 
.9  45.33  6;2 

,  7/  34 .86  ,6.1 
•42  38.19  5.9 

t'  , "  . 

f '38*53  '6.6 
40.00  7.6 

38.56  .5.8 
,  21  36.76  5.8 

■ 

•3* '38.83  '  ft 
<11'  40.36  8,0 

16  36.44  10>7. 


31  32.90  13.5  ,  11  40.73.  6,6 

15  54 
4  21.00  .  5.f 


44  13.84y  3.4 
20  13.85  \  3.5 
13.83  J3. 


24 


4     .13  20.62  5.3 


74  14.70  2.9 
34  15.32  3.1 


3?  mM  1 6.** , 

,6  $5.67  8.0' 
40  14.17    2.6  '  .76  18.62  . 5..3.  ; 


*  '5!  m 


230 


Table,  5.2 


V 


Pretest-Postteat^baerVations 


v  Postteat 

N  *£an  ii 

52  39.27  '  6.8' 

9  45.33  6.2 

4  / 34. 75  7.9, 

,  W  38.33/  6.1{ 

;  •  f 

46  34.09  ^  ,9.8  v  '  46  39.07  6.6 

21  40.86  .6.8 

10  39.10  v6.8 

15  36.53  6.0 


Pretest' 
I  .  Mean  ./  ad 

*  52  38.19  10.6 

9  47.22  10.9 

4  36.50  .14.8 

39  36.28  9.2< 


21  34.33/10.0 
10  34.0JI  11.0 
15  33.80.  "9.7 


32.  3f  37  13.-2  '  32  38.56  9.5 

9  /1.33  14.1  ^  9  40.56  8T.7 

33.21  12.^  „  -14  35.79  11.3 
9  43.33'.  12.2 


14  14.29  ,3.2 
1  u.OO.   ■ — 
.  13  14.54  .3.2 

30  14.80  2.9 
6  17.50'  ,1.8 
'  $4.1**42  2.« 


9  40.8,9  -7.4 

.14  20.93  ,5.2. 

1  25^00  — 

13  20.62  5.3 

30  19«83  6.4 

6  25.67  "8.0 

24  18.37  .5.3 


\ 


Another  ^>oint  evident  from  the  summaries  of  achievement  data  i8  the 
,  superior  pbsttest  perf  ormanc«j  of  students  under  the  TICCI?  condition.  The 
average  posttest  score  for  students  in  the  TICCIT  .program  was  at  least  one-half 
i      'of  a  standard  deviation  higher  than  the  average  posttest  score  of  students  in 

"Mb 


lecture  sections This  difference  held  across  courses-  in  the  fall  term,  but 
occurred  in/fewer  courses  in  subsequent  terms :  '  half  of  a  standard  deviation 
separated  the  conditions  during  the  fall  term  for  Math  007  and  108  at  Phoenix 
and  Math  31  at  Alexandria,  but  the  mean  posttest  scorefe  for  TICCIT  arid  lecture 
classes  were' much  ^closer  in  the  spring  term  for  these  three  courses.  ,  The 
apparent  inconsistency  of  achievement  results  across  lierms  suggests  a^re levant 

question:     Why  was  there  a  pronounced  difference  in  posttest  performance  in  the 

/       *  ii    '  \*  " 
w..  taii  term  land  not  in  later  terms? 

Pretest  scores  probably  contributed  to  the  observed  differences  in  post- 
test  performance.    Larger  differences  in"  phe  mean  pretest  scores  of  students 
completing  a  course  tend  to  be  associat^  with4arger  differences  in  posttelst 
performance  between  conditions,     m  Beginntag  Algebra  at  Phoenix,  'for  example, 
"  the  fall  pretest  means  for  students  completing  either  the  TICCIT  program  or 
n      lecture  instruction  were  44.86  and  41.28, ^respectively.    The  corresponding 

spring  pretest  means  were  40.06  and /38.£2;,'  There  was  less  of  a-  difference  & 

between. conditions  iri;.terms  of  both  pretest  and  posttest  performance  in  the 

♦     «/  9  .       •  ■   /  ■*'/?''' 

spring  term.    And  ttfe  notable  decrease  in  the  fljean  pretest  score  for  thje  TICCIT 

condition,  44^66  jfn  the  , fall  to,, JO. 06.  in  the  spring,  is  explained  *by  the  pre- 

test  scores  of  |3  students.    There  were  13  students  who  had  completed  Math  > 

^tW/ -with  TICCIT  in  the  spring  semester  after  starting  the' course  in]  the  f  ai  lfo  2 

tjielr  mean  pretest  score  from-  the- fall  was  33.92;  nearly  one  standard  deviation 

•  below  that  of  students  completing  a  math  course  V TICCIT  within  one  term.  By 


contrast,  just-eight  of  the  ^40  pretest-posttest  observations  fof  the  lecture 
condition  represented  students  repeating  *iath  0&7;^eir  mea*  pretest  score  s 
*rom* the  -fall  was  34. 62.     It" seems  reasonable  to  conclude  that  8tudents  who 
took  lon^*  Cha%n  one  academic,  term  to  complete  their  course  studJ,e8  affected 
the'deecriptive  test  statistics  for  later  terms  in  the  SvalUat ton>  especial1^ 

f      M  _  * 

within  the  TlCCIT  condition  where  such  students  constituted  {tom  one^f purl* 

»  •.  1  ' 

to  one-half  of  the  total  number  of  students  completing  a  c°UcSe  in  later  term8'. 

There  were  exceptions  to  the  general  pattern  of  results  scross  terms.  ^he 

spring  achievement  data  for  Math  106  still  indicate  a  sizeab^  gflp  in  posttest 

performance  by  condition,  and  nearly  two-thirds  of  a  stands^  d^via£ion  8eparate 

the' conditions'  pretest  means        both  semesters.    The  expla^^*  i8  8imply  that 

the  pretest  for  Math  106  in  the  spring  was  different  from-  the  Qne  administered 

'  f  *  ^  *  for  '  " 

in' the  fall;  the  spring  pretest  was  to  serve  as  both  a  measUte  of.  re^entio11  or 

students  who  had  taken  Math  007  in  the  f^ll  and^  elected*Math  XQ6  a8  ^  ne*t 

course  in  an  algebra  sequence  as  well,  as  a  measure  of  entranQe-  abildty.  This 

1  chan^  in  pretest  forms  precluded  taking  an  earlier  pretest  as  a  ^tch  for  a 
later  po§Jt test:     all  pretest-posttest  observations,  had  to  b*  taken  from  with±n 
the  spring  semester.    This  alsqn  led  to  a  poor  percentage  of  Postte8t  observations 

•paired  with  pretests  in  the  T ICC IT  condition  (29%).  .  *  - 

At  Alexandria  ten  students  who  completed  Math  31  undef  ^  TICC1T  pro&ram  in 

'  *i     '  •  '  f  11 

the'winter  quarter  had  actually  started  the  course  and*taken  pretests  in  the    a  . 

Again  there  was  a  marke^decrease  in  the  mean  pretest  score  fQr  8tudents  com- 
pleting the  course  under  the  TICCIT,  condition,  from  47.22  l^  ^  fall  to  34.33 
in  the  winter.    Unlike  other  cases  )md  despite  the  si^ari^  q£  prete8t  scores 
across  conditions,  p^potittest  means  for  Math  31>n  the  wl*ter  quarter  ■til^  .  | 
'reflect  an  advantage  for  -students  completing  the  course  $.th  TlCClT.    Posttest;.  J 

i     292        .  V  .  •  I, 

•  '  >  '  ■Ill 
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performance  for  Math  31  in  the  spring  quanjer  was  comparable  in  the  TICCIT  and 
lecture  conditions,  as  was  pretest  performance.    It  is  difficult  to  discern 
any  pattern  in  the  achievement  results  for  T1CC1T  in  relation  to  programmed 
instruction  since  there jwere  £o  few  observations. 

Closer  inspection  jbf  the  achievement  data  can  reveal  further  interesting 
features  of  the  results!.    For  example-,  the  mean  pretest  scores  of  students 
starting  a  course  at  PlWnix  varied  littl^according  to  condition.  Perhaps 
pretest  means  across  condfte^ons  were  even  closer  in  the^Pring_^eme8ter"  than 
the  $£11.    Yet  students  chose  their  own  classes  in' the  spring  while  they^  " 
had  been -randomly  assigned- to  conditions  in  the  fall.    At  least  in  terms,  of 
entrance  ability, . student  selection  and  random  assignment  Were  equivalent    5  •' 
^  methods  for  forming  comparison  groups.  _  fcv    *  ... 

But  the  central  issue  suggested  by  the  Achievement .data  is  whether  entrance 
'  \  *  '  •    /  - 

ability  accounts  for  the  higher  posttest  performance  observed  under  the  T1CCIT 

program.    Analyses  for  the  math  achievement  results  should  provide  an  answer. 

•  .       '.  .  .  ■  \  1  •  '  ; 

5.2.b     English  Courses  C  \    f  t  s  < 

Summaries  of  achievement  data-  f rom  English  eoursVs  appear  in  Tables  5.3 
and  5.4.    Th*se  tables  follow  the- same  format  as  earlieV'math  summaries,  except 
phey  present  descriptive  statistics  for  two  ^fpes  of  tests,    T£e  pretests"  and 

\posttestsyg*ven  in  English  courses  included  both  a*f  object £ye  test „o$  writing 
Skills  andean  essay  test.    Again  the  data1  for  all  pretests  refer  ,to  the  strident 

.population  which  started  a  course.    The  information  tor  all  p'oatteets  reflects 
the  performance  of  those^tudents'  who  completed  a  course;    D^&iptive  test"  j. 
Statistics  f oa:  pretest-posttest  observations  also  reJresetitNfhe  performance  of  C 
those  students -Who  completed  a  course,  and.  thus  give pretest ;.4ata 'for  just  those 

-~  ■  '       ■ "     v  <v   ■»  *   ;  *     ■        - ':  'V   '      •  \j . 

student*  rather  than  the  entite^tudent  population  that  started  a0  course. 


Achievanant  Su«ary 

Phoenix 
,    English  19  and  29 
CalMuUr  Year  1976 


Objective  Teat  of  Writing 

*"  .  Comparison 

Terra . 


Skills 


Spring 


V  4 


Fall 


Total 
TICCIT 

rtCCIT  Q9); 

TICCIT  (29, 
Lecture 
Lecture,  49) 
Lecture  $29) 
Day  (19) 
Evening-  (29) 


Total 

ticcit;'  . 

-  TICCif  (19)  " 
TICCIT  (29f 

"  Lecture, (19) 
Lectv4re\(29)-^ 

,Day,,il5),?,  r* 
Ev^tng^29)^ 


(  Essay  Tests1 
Spring. 


Tot4 
TICGIT 

T*CCIT  (19) 

TfcCTJ.  (29) 

Lecture 

Lecture  (19) 

Lecture  (29) 

(19)  C 

Evening  (29) 


All  Pretests 
N    Mean  s'd 


All  fosttests . 
N    Mean  &d 


Pretest-Posttest  Observations 
Pretest  Post  test 


Mean 

.J  « 

3d 

M 

N 

Mr. 

SQ 

24.49 

6.6 

131 

26.73 

6.0 

24.30 

6.3 

47 

2;.  81 

5.4 

23.33 

&»3  . 

11 

H.J 

Zo 

00  70 

Zo.  /J 

0  •  i 

f  Oi  / 

Qf 
04 

O.J 

H3.*fOT 

»  A  0 
0*7 

JH 

OQ  flA 
Zj  .UU 

O.J 

7b. 

J. 7 

jU 

Zo.ij 

5  ft 

J  .0 

01   /.  A 

0.  / 

"7S 

OS  Ll 
Zj#H/ 

o  it  on 
25.89 

0.1 

JO 

OQ   y  1 

j*y 

oo  40 

Z/o 

ZO.^.0 

0.1 

25.37 

7.3 

87 

27.53 

5.9 

24.^6 

7.'7 

59 

27.64 

5.4 

27.50 

6.1 

28 

27.29 

7.0 

22.82 

7,3 

189 

25.67 

6.1 

22,14 

6.% 

145 

25.25 

6.0 

25.05, 

8.3 

44 

27.07 

6.5 

22.7$ 

7.2 

204 

25.94 

5.9 

26.00 

7.6 

72 

27.15 

6.7 

244' 

1.96 
2.12 


.7 
'.8 


*  J  j  iff 

45* 

2.14 

.8 

2.08 

.8 

137 

2.14 

.8 

49  ' 

2.26" 

.7 

V47 

2.10 

.8 

47 

2.26 

.7 

21' 

2.>6 

.7  > 

*  21 

1.74 

.6 

21 

2.36 

.7 

28 

2.1(8 

.7  ^ 

26 

2.38 

.§ 

26 

2.17 

.7 

96 

2.08 

'.8 

90 

2.08 

.8 

90 

2.08 

.8 

06 

2.14 

.8 

62 

1.98 

.8 

62 

2.15 

.8 

30 

1.95 

.7 

28 

2.29 

.7 

28 

1.93 

.7 

87 

2.20 

.8 

83 

1.92 

.8 

83 

2.20 

.8 

58  ' 

2.06 

.7 

54 

2.33 

.8 

54 

2.05 

.7 

Eall      ^Tctal   .  * 
TICCIT 
\  .TICCIT  (19)  * 
/ TICCIT  (29) 
/  Lecture 
^  Lecture  (19) 
Lecture  (29) 
Day  (19) 
Evening  (29) 


ERICy 


i  506 

f  20b 

f  147 
J  61 
'  298 

210 
88 

35? 

149 


1.91 
1.88 
1.79* 
2.07 
1.94 
1.87 
2.11 
1.8 
2 


L.84 
'.09 


'  .7- 
v".7 

>7 
<.8 

.'7 

.7 

.8 

.7 

.8 


Table  5.3 

i  1 


320 

2.09 

.7 

297 

1.99 

.7 

297 

2.08 

.7 

117 

2.08  . 

99 

1.95 

.7 

99 

2.07 

.7 

77 

2.14  f 

.7 

74 

1.92 

si 

74 

'  2.14 

.7 

40- 

1,96 

.7 

25 

■  2.04 

25 

*1.&6 

.8 

203 

2.10 

.7 

198 

2.01 

..7 

*198 

2.09 

.7 

157 

2.03 

d 

154 

1.94 

.7 

154 

2.02 

.6 

46 

2.34 

.8 

44 

2.26 

.8 

44 

2.34 

.8 

234 

2.06 

.7 

228 

1.93 

.7 

228 

2.06 

.7 

86 

2.16 

.8 

2.18 

.8 

69 

2.17 

.8 
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Achievement  Summary 
Alexandria 

•  English  111 
Cal jndar  Year  1976 


Objective1  Test  of  Writing  Skills 


Term 


Winter 


Spring 


Fall 


Essay  Tests 
Winter 

Spring 

Fall, 


Comparison 
Group 

Total 

TICCIT 
Lecture 

Total 

TICCIT 

Lecture 

Totel  \. 

TICCIT 

Lecture 


Total 

TICCIT 

Lecture 

Total' 
TICCIT 
Lecture 

Total 

TICCIT 

Lecture 


All  Pretests 


All  Posttests 


Mean 

sd 

N 

Mean 

sd 

184 

28.55 

6.7 

96 

30.98 

6.0 

102 

30.05 

5.5 

57 

32.18 

5.6 

82 

26.70 

f 

7.6 

39 

29.23 

6.0 

312 

26.56 

8.8 

248 

28.31 

8.7 

116 

21.66 

8.8 

84 

22.83 

9.2 

196 

29.46 

7.4 

164 

31.12 

7.0 

259 

25.59 

7.4 

206 

28.78 

6.4 

163 

25.60 

7.3 

122 

28.75 

6.7 

96 

25.58 

1a 

84 

,  28.82 

5.9 

176 

,2.67 

.6 

96 

2.46 

.8 

95 

2.71 

.8 

58 

2.72 

.8 

81 

2.63 

.7 

38 

2.07 

.8 

311 
119 
192 


2.51 
2.15 
2.73 


.8 
.6 
.7 


.7 
.8 


248 
83 
165 

196 
113 
'83 


2.49 
2.10 
2.68 

2.36 
2.42 
2.27 


Me- 
ltable 5.4 


.8 
.8 
.8 

.8 
.8 
.8 


N 


90 
54 
36 

236 
82 
154 

183 
107 
76 


Pretest-Posttest. Observations 


Pretest 
Mean  sd 


93  29.09 

'55  31.44 

38  25.68 

236  26.68 

81  20.68 

155  2P/82 

192  25.70 

117  25.96 

75  25.29 


2.63 
2.75 
2*46 

2.56 
2.15 
2.79 

2.14 
2.11 

J'18 


6.3 
4.4 
7.0 

8.9 
8.5 
7.3 

7.3 
7.3 
7.4 


Posttest 
N  Mean 


93 
55 
38 

236 
31 
155 

192 
117 
75 


,31.38  5.6 

32.65  5.1 

29.53  5.8 

2o\35  8.8 

22.64  9.3 

31.34  6.9 

28.68  6.4 

28.63  6.7 

28.76  6.0 


.8 

90 

^2.46 

.8 

.8 

54 

2.71 

.3 

.7 

36 

2.08 

.8 

.8 

236 

2.48 

.8 

.8 

82 

2.09 

.8 

.7 

154 

2.69 

.8 

.7 

183 

2.34 

.8 

.7 

107 

2.38 

.7 

.8 

76 

2.29 

.8 

ERIC 
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1  *  i  • 

Two  points  on  the  objective  tests  of  writing  skills  or  two-tenths  of  a 
point  on  the  essay  tests' would  indicate  a  difference  in  scores  of  five  percent 
of  the  tot^l  possible  score*     Such  a  difference  on  either  type  of  test  would  be 
equivalent  to  about  one-fourth  of  a  standard  deviation.    Differences  of  this 
size  were  common  for  student  achievement  in  math  courses  but  seem  tare  in 
English  courses,  ^ 

Even -higher  pretest  means  for  students  completing  a  course  as  compared  to 
all  students  starting  a  course  were  infrequent.     English  029  is  the  only  case 
that  clearly  demonstrates  a  link  between  entrance  ability  and  course  completion. 
But  entrance  ability  seldom  contributed  to  the  explanation  of  completion  rates 
in  English  courses  (see  Tables  4.8  and  4.9)  and  When  it  did  represent  a  signi- 
ficant explanatory  variable,  it  was  as  part  of  a  trait- treatme&>  interaction. 
These  later  instances,  English  111  in  the  winter  and  fall  quarters,  were  cases 
in  which  students  completing  a  course  with  TICCIT  had  slightly  higher  s?ores 
on  the  objective  pretest  than  all  students  who  had  started  the  course  in  TICCIT 
classes  (e.g.,  31.44  and  30.05  in  the  winter  quarter)  while  students  completing 
the  course  in  the  lecture-discussion  condition  had  slightly  lower  scores  than 
all  students  who  bad  started  under  that  condition  (e.g.,  25.68  and  26.70  in  the 
winter  quarter).     Although  there  was  evidence  of  a  differential  treatment  effect 
on  course  completion  (see  Chapter  4:     Course  Completion  Rates),  it  apparently 
was  a  subtle  phenomenon  and  should  not  invalidate  the  achievement  analyses  for 
English  courses. 

The  most  obvious  differences  in  test  performance  arose  in  Alexandria. 
During  the  winter  quarter  the  mean  scores  associated  with  the  TICCIT  condition 
were  about  one  standard  deviation  higher -than  thos^  for  lecture-discussion 
instruction  on  both  t^e^objective  pretest  and  the  essay  posttest.     The  objective 

236 
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posttests  and  essay  pretests  reflected  differences  in  the  same  direction  though 
not  of  the  same  magnitude.    Again  in  the  spring  quarter  there  were  gross  dif- 
ferences  in  test  performance  by  condition.    The  mean  scores  of  the  two  comparison 
g^ugsJwere  about  one  standard  deviation  apart  on  all  tests.     But  the  differences 
were  in  the  opposite  direction.    During  the  spring  quarter  the  better  students 
in  terms  of  entrance  ability  were  in  lecture-discussion  classes.    Evidently  the 
comparison  groups  in  these  two  terms  were  not  equivalent  at  the  outset  of  the 
course.    This  fact  is  not  so  much  a  function  of  inadequate  evaluative  controls  ^ 
over  group  composition  as  it  is  a  result  of  a  conscious  effort  by  the  college  to 
try  the  TICCIT  program  with  students  of  different  abilities.     By  the  fall  quarter' 
arrangements  had  been  made  for  a  stronger  test  of  thg  TICCIT  program  as  an  ad- 
junctive  resource  integrated  with  classroom  practices.     The  descriptive  test  data 
for  that  quarter  suggest  that  comparison  groups  were  similar  in  terms  of  their 

entrance  ability,  and  data  on  the  time  students  spent  on  the  TICCIT  system  (see 

i 

Chapter  7:     Student  Activities)  indicate  extensive  use  of  the  TICCIT  progftim. 

Data  from  the  fall  semester  at  Phoenix  also  reveal  a  sizeable  difference  in 
test  performance.     Students  completing  English  029  on. the  TICCIT  system  had  an 
average  score  on  the  essay  posttest  of  1,86  while  the  average  of  students  from 
lecture-discussion  classes  was  2.34.     Entrance  ability  as  reflected  by  the  essay 
pretest  probably  accounts  for  some  of  this  gap  on  the  essay  posttest.     It  seems 
to  us  more  likely  that  the  part-time  instructors  responsible  for  TICCIT  classes 
that  semester  were  unfamiliar  with  the  program  and  unable  to  take  advantage  of 
.its  strengths.     This  disparity  in  essay  posttest  scores  can  be  attributed  to 

-  / 

instructor  effects  rather  than  the  treatment. 

Other  differences  between  pretest  and  posttest  performance  might  be  noted, 
but  this  would  be  an  improper  interpretation  of  the  data  since  the  English 
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pretest  and  posttest  were  separate  instruments.     Score  differences  between 

pretest  and  posttest  do  not  reflect  gains.    Nor  does  it  seem  appropriate  to 

belabor  differences  across  courses,  despite'  the  use  of  the  same  instruments 

as  pretests  and  posttests.    The  English  courses  at^the  two  colleges  were 

intended  to  serve  different  student  populations  and  had  different  emphases. 

But  the  achievement  results  for  English  courses  do  require  analyses  to  make 
t 

effects  clear. 

•s 

5.3    Analysis  of  Program  Effects 

The  achievement  data  from  pretesting  and  post-testing  leave  the  issue  of 
TICCIT's  impact  on  cumulative  learning  unsettled.     Simple  inspection  of  the 
results  does  not  rule  out  the  possibility  that  entrance  knowledge  accounts 
for  the  generally  higher  posttest  performance  of  students^completing  math 
courses  under  the  TICCIT  program.     Similarly,  the  differences  in  test  perfor- 
mance observed  in  English  courses  seem  to  be  too  small  to  draw  conclusions 
from  the  data  alone  about  the  effectiveness  of  TICCIT  instruction  relative 
to  lecture-discussion  practices.     So  we  must  turn  to  analyses  of  these  data 
in  order  to  determine  whether  there  were  significant  treatment  effects  on 
student  achievement* 

<* 

Often  an  apparent  treatment  effect  can  be  attributed  to  sources  otfier 
than  the  program  und^r  study.     It  may  be  some  student  characteristic  (e.g., 
entrance  ability)  or  a  variable  associated  with  particular  classes  (e.g.,  the 

instructor)  that  accounts  for  differences,  in  posttest  scores  seemingly  due  to 

/ 

the  instructional  condition.     There  may  also  be  characteristics  of  the  tests 

i  * 
themselves  that  disguise  treatment  effects  (e.g. ,  posttest  coverage  weighted 

toward  one  program  or  pretest  reliabilities  indicating  a  greater  *rror  of 


IRJC  .    ..  ^38 


v. 


-  261  - 

measurement  within  one  comparison  group) *     Even  further,  data  taken  from  a  few 
academic  terms. for  several  courses  may  reveal  an  effect  peculiar  to  the  con- 
•ditions  of  one  specific  term  or  the  materials  appropriate  to  one  particular 
course.     All  of  these  considerations  enter  into  our  analyses  for  student 
achievement  so  that  we  might  isolate  and-estimate  TICCIT  effects. 
\ 

5  •  3  •  a    Math  Courses  ' 

The  posttests  prepared  as  measures  of  student  achievement  in  math 
courses  actually  gave  a  number  of  different  subscores.     The  basis  for  our  com- 
parisons of  TICCIT  results  with  those  of  regular  classroom  practices  was  the 
subscore  that  reflected  course  objectives  shared  by  both  conditions.  This 
subscore  for' common  objectives  typically  included  about  85  percent  of  the 
iteiis  on  a  course's  posttest.     The  other  items  concerned  objectives  unique 
to  either  the  TICCIT  program  or  lecture  classes,  and  often  two  or  three 
questions  went  beyond  the  course's  objectives  regardless  of  condition.  The 
analyses  presented  here  deal  first  with  common  objectives  and  then  with 
objectives  unique  th  one  condition.  %* 
It  may  be  recalled, that  our  approach  to  data  analysis  involved  estimating 
treatment  effects  for  similar  students.     Similarity  was  to  be  defined  by  a 
number  of  demographic  variables,  such  as  a  student's  entrance  ability  in  the 
subject  matter  and  age.     Requiring  these  additional  data  meant  the  loss  of 
some  observations  in  the  analysis  due  to  incomplete  information.     This  happened 
in  the  analyses  for  course  completion,  but  the  number  of  observations  was  so 
large  that  we  could  afford  the  loss.-    With  the  number  of  pretest-posttest 
observations  already  low  for  math  courses  under  the  TICCIT  program,  we  had 
to  retain  all  of  the  observations  and  chose  not  to  sacrifice  any  for  missing 
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demographic* data.  'The  variable  that  def  ined  s8tudent.  similarity  in  the  analyses* 
for  achievement  in  math  courses  was  entrance,  ability  as  measured  by  our  pretests. 

Tables  in  this  section  present  the  analyses  for  student  achievement  as  a  . 
program  outcome  in  math  courses.    Each  coursa  appears  alpng  with  a. simplified 
regression  model  for  the  posttest  score  under  consideration.    That  score  may 
correspond  to  the  number  of  items  correct  related  to  common  objectives,  a  par- 
ticular ability  level  (e.g. ,  factual  recall,  solution  of  routine  problems),  a 
specific  content  category  (e .g.  /  arithmetic,  algebraic  equations,  number  line 
and  coordinate  plane) ,  or  objectives  unique  to  one  condition,    whatever  the 
particular  subscore  represents,  it  is  part  of  the  posttest  and  therefore  serves 
as  the  dependent  variable  in  the  regression  model.    Usually  just  two  or  three 
terms  appear  as  independent  variables  in  the  regression.     One  is  for  entrance 
ability  and  defines  similar  students  under  each  condition  through  their  pretest 
scores.    Another  might  be  a  simple  dichotomous  variable  representing  two  dif- 
ferent divisions  (i.e.,  day  and  evening)  or  terms  (i.e.,  winter  and  spring). 
And  there  is  always  a  term  representing  the  instructional  conditions.  The 
regression  coefficient  for  this  term  is  the  estimated  treatment  effect,  T. 

Common  Objectives.    Tables  5.5  and  5.6  present  the  analyses  for  program 
outcomes  in  terms  of  posttest  scores  on  common  objectives.     In  contrast  to  the 
analyses  for  course  completion,  the  TICCIT  impact  on  student  achievement  tends 
to  be  positive.     Eight  of  the  11  cases  presented  in  th^  tables  show  that  students 
completing  math  courses  under  the  TICCIT  program  had  higher  posttest  scores  than 
similar  students  from  lecture  classes.    A  statistically  significant  difference 
was  associated  with  two  cases:     Math  007  in  the  fall  semester  at  Phoenix  and  Math 
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fchliuunt  Imlti:  Analysis  on  Com  Objietlvu 
Ptotaix  Kith  Counts 
teiimk  Tut  1973-76 


Course 
Tin 

Nuaber  of  Students 
Multiple  Correlation 
Standard  Error  of  Estate 

\ 

Independent  Variables 

Condition: 
TICCIT(1)/Lecture(0) 

Entrance  Ability; 
Pretest 

Division { 
Day(l)/Evening(0) 

Constant 


Kith  007 
?all  Semester 
N  -  152 

sV6.01  ■ 

Regression   Value  of 
Weight  t-statiWc 

2.87  2.05* 

.53  11.56*** 

4.30  3.94*** 
8.80 


Math  106 
Fall  Semester 
11-71 
R  -  .45 

est 

Regression  Value  of  . 
Weight  t-statlstic 


2.18 
'  .23 

15.50 


t.71 
3.12*** 


Hath  108 
Fall  Semester 
N  -  47  . 
R  •  .47 

"est"7'63  V. 

Regression  Value  of 
Weight  ^statistic 

3.24  ,  l.oi 
.35  3.04*** 

16.21 


Course 
Term 

Umber  of  Students 
faltiple  Correlation 
Standard  Error  of  Estimate 

Independent  Varlablea  . 

Condition: 
TICClT(l)/LectuTe(0)  ■ 

Entrance  Ability: 
Pretest 

Division: 
Day(l)/Evening(0) 

Constant 


Math  007 

Spring  Semester 

N-  172 

R  -  .56 

se^  •  6.02 
est 


.'  Math  106 
Spring  Semester 
M  -  91 
R  -  .67 

8V 5-87 


Math  108  1 
Spring  Semester 
'  N  -  52 

R-  .34 

est 


Math  117 
Spring  Seae8ter 
H  -  64 
R  "  .73 

"est' 


Regression  Value  of  Regression  Value  of  Regression  Value  of  '  Regr€C6l0„  V]tnf 
J^t^tatisdc    Jt^    j^^  JyCtSSti 


.36  .29 
.38  8.88*** 
2.40* 


2.37 
17.63 


1.50 

1.95 
3.97 


.26 

7.60*** 
1.47 


-.17 

.20 

-4.56 
29.48 


-.07 
1.92 
-1.53 


-3.05 

.51 

10.35 
8.17 


statistic 

-1.72  • 
2.43* 
6.07*** 


*P<  .05 
**p  <  .01 

***p  <  .001 


Table  5.5 
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Achievement  Result! :  Analysis  on  Coonon  Objectives 
Alexandria  Math  Couraea 

Academic  Tear  1975*76  ' 


Course 
Term  * 

Number  of  Students 
Multiple  Correlation 
Standard  Error  of  Estimate 

Independent  Variables 

Condition; 
nCCIT(l)/Lecture(0) 

Entrance  Ability: 
Pretest 

Term: 

Winter(l)/Spring(0) 

Constant 


Math  31 
Fall  Quarr.er 
N  »  48 
R  -  ,55 


se    -  4,49 
est 


Math  31 

Winder  Quarter 

N  -  36 

R  -  M 

*e    ■  5.15 
est 


Math  31 

Spring  Quarter 

N  ■  18 

R  -  .64 

se   .m  4.95 
est 


Math  32 
Winter  &  Spring  Quartan 
K  ■  44 
R  -  .51 
.seesc-4.62 


Regression    Va]ue  of     Regression    Value  of  Regression     Value  of  Regression  Value  of 

Weight     t-statistlc     Weight     ^statistic  Weight  t-atatiatic     Weight  t&tatlstic 

'  •  v. 

1.79            .98          2.91           1.67  ■  -  .93  -  .40  4.31  2.15* 


.'24 


24.37 


3.51 


** 


.22 


2.43 


.30 


22.93 


3.15 


.56 

1.99 
7.45 


2.27 
1.31 


p  <  .05 


p  <  .01 
***p  <  .001 
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Table  5.6 


32  at  Alexandria.    Both  involve  a  point!  difference  in  favor  of  the  TICCIT 
program.    There  were  three  cases  of  a  negative  treatment  effect,  but  none  of 
these  was  significant  at  accented  levels  of  probabili^. 

These  results  hardly  seem  overwhelming.     It  might  be  said  , instead  that 

at 

nine  of  11  cases  showed  no  significant  difference  in  student  achievement. 
That  would  seem  ; to  be  a  fair  interpretation,  especially  if j  some  alternative 
explanation  exists  for  the  significant  positive  effects.    For  Math  32  at 
Alexandria  there  were  very  few  pretest-posttest  observations  in  the  TICCIT' 
condition.     The  seven  students  representing  that  population  which  received  a 
passing  grade  for  Algebra  II  on  the  TICCIT  system  would  certainly  temper  our 

interpretation  of  the  educational  significance  of  the  treatment  effect. 

? 

Indeed,  an  analysis  conducted  for  all  students  posttested  regardless  of  grade 
would  indicate  an  effect  in  the  opposite  direction.    The  treatment  effect 
would  then  be  -1.35  and  its  associated  t-statistic  -.99  (based  on  32  obser- 
vations within  the  TICCIT  condition  and  47  within  lecture).     Hence,  the. 
difference  in  posttest  performance  goeS  against  the  TICCIT  condition  for 
students  attending  classes  for  Math  32. 

The  other  case  of  a  significant  treatment  effect  was  Math  007  in  the  fall 
semester  at  Phoenix.     Since  the  pretest  reliability  for  students  with  post- 
tests  was  .83  within  the  TICCIT  condition  and  .93  within  the  lecture  condition, 
perhaps  the  treatment  effect  was  an  artifact  of  a  test  characteristic.  But 
the  analysis,  corrected  for  pretest  reliability  (after  Porter  &  Chibucos, 
1974),  would  yield  an  even  higher  estimate  of  the  treatment  effect,  t  -  3.13 
and  t  *  2.14,  £  <  .05.     Similarly,  the  pretest-posttest  correlations  varied  so 
much  by  condition,  .46  within  the  TICCIT  condition  and  .78  within  lecture,  that 
this  might  be  seen  as  a  strong  competing  explanation  for  the  apparent  treatment 
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efcfect.    Yet  we  should  Expect  such, a  reduced  pretekt-posttest  correlation"  within 

*=**.«  '  * 

the  TICCIT  condition  given  the  restricted  pretest  rajige  among  students  completing 
a  math  course,  ui(der  the  TICCIT  program.     If  we  accept  this  experimental  effect 
on  course  completion  rates  and  correct  the  analysis  for  unreliability  according 
to* test-retest  correlations  (after  Campbell  &  Bo rich,  1975),  tjje  estimated 
treatment  effect  (x  =  2.75  and  t  -  1.88)  would  still  be  consistent  with  our 
original  estimate  in  Table  5.5  (x  -  2.87  and  t  «  2.05).    Nor  is  the  effect 
simply  a  matter  of  systematic  biap  due  to  postt^st  observations  without  pre- 
tests.    Adjusting  for  missing  pretests  would  reveal  a  somewhat  higher  treatment 
effect,  x  *  3.30. 

It  might  still  be  argued  that  entrance  ability  accounts  for  the  better  Per" 
formance  of  students  in  the  TICQIT  condition.    To  some  extent  it  does:  almost, 
every  analysis  indicates  that  pretest  score  contributes  significantly  to  the 
explanation  of  posttest  variance.    But  with  entrance  ability  taken  into  con- 
sideration there  was  a  significant,  treatment  effect.     Even  comparisons  of  just 
those  students  with  similar  pretest  scoresV would  indicate  that  the  TICCIT  program 
has  a  positive,  effect  on  cumulative  learning.     Such  an  analysis,  ftfr  Math  007 
in  the  fall  semester ,  with  the  range  of  pretest  scores  in  the  lectu^ng^condition 
restricted  to  X  >_  35  as  an  approximate  match  to  observed  scores  in *We  TICCIT 
condition,  would  result  in  an  estimated  treatment  effect  of  2.37  and  a  t-statistic 
of  1.93.    The  positive  TICCIT  effect  for  Math  007  in  the  fall  semester  holds  under 
a  variety  of  analyses. 

Actually  the  treatment  effect  evident  in  Math  007  for  the  fall  semester  was 
even  more  dramatic  than  the  estimates  thus  far  presented  suggest.    T^ere  was  a 
statistically  significant  interaction  term  omitted  from  the  simpli^^d  regression 

model  in  Table  5.5.     It  indicated  that  the  treatment  effect  was  different  in  the 

,  ) 

3f;S 


flay  atnd  evening  divisions.    Although  the  point  difference  in  postte8t  perfar-  V; 
mance.  by  condition  was  positive  for^the  day  division,  TD  -  1%£     >ttfie  e^tix^ted 
treatment* effect  for  the  evening  division  was  quite  pronounc^  1  ^ v  -  8.43  *nd 
t  -  2.99,  £  <  .01.     Sample  size  in  the  evening  division  for  t^  TlcclT  c6hdition 
(N  -  9) ^prec^ded'  reliability  adjustments  within  groups  def         ^  bQ(_h  ^dition 
and  division.    However,  if  we  were  to  include  a  treatment^  k  dtvisipn  term"  in  our 
analysis  corrected  for  test  unreliability,  we  would  find  both  the  adjustment  for^ 
pretest  reliability  and  that  for  teist-retest  correlation  8h0v  ^  siaii^  •eBtH"®^-" 
^treatment  effect  for  the  evening  division  (t£  -  8.46  and  fe' *  3(0^£  <    01,  »nd 

TE  "  8,59  and  C  "  3'06»  £  <  -01'  for  the  two  methods,  re8p®cttVely) .  'These 
results  conform  with  our  findings  for  Math  007  across  semest^  (8ee^Table  -.5.8) 
as  well^as  the  appearance  of  the  data  (see  table  5.D-  they  lend  further 

weight  to  the"  interpretation  of  the  TICCIT  effect  onjstudent  QChlevement  a8  ' 
positive  and  significant  for  mainline  applications  in  math  c0ll^  .  * 

This  evidence  of  a  treatment  effect  should  convince  us >tQ ■•  look  elsewhere  .q 
for  supporting  results.     The  analyses  presented  in  Table  5.5  £0r  thtL  8prij^  ,'i'v 
semester  at  Phoenix  included  only  those  students  who  complete^         r  COurfle  V. 
studies  by  the  end  of  the  academic  term.     Yet  there  were  some        dents  whP 
took  advantage  of  the  self-paced  nature  of  the  TIGCIT  program  ^  c0ncentrate4 
on  other  courses  instead  of  completing  their  math  course  on  t^e  TlCCllv  system. 
Later  they  did  fulfill  their  math  course's  requirements.  InciUding 
students  in  the  analyses  for  the  spring  semester  would  give  Us  mQre  obser-V 

vations  in  both  Math  106  and  Math  108,  a  substantial  increase  glven  the  1qW 
sample  size  for  the  TICCIT  condition  in  these  courses.     Also,  collection 
for  Math  106  and  Math  117  continued  into  the  summer  semester;  ^  additional 
observations  for  the  TICCIT  condition  in  each  of  these  course^  become  available. 
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Analyses*  which  include  studentSa  completing  a  math  cotfra£  lifter  the  formal 
Close  of  the  spring  semester  or  during]  the'  sumtfter  sem^&er  appear  intfable  5.7. 
Thete  would  be  only  two  additional  db^rvatMp  itx  ^W607  for^TICCIT  comple- 
tions  immediately  after  the  spring  sternest  e^j^^ "  t|^^^ourserfJjB*^fa±tted  from 


,the  table.     For  the  three  co ur^e s  wh ^^^^n?^^^e r e  is  *n  incr^se 

estimated  treatment  effects  ^^^^  posttest  data  col-iec4ed 

'within  the  spring  semelter^^^  a ^significant,  positive 

TICCIT  effect  fpr  Hath  ^06  Math  106  includi 

si*  students  from  the  TJCC^K  coitf  1$W  in  the  summer  semester  but  no  students 
from  lecture  classes  in  that  te^^thfere  is  season  to  question  the  estimated 
:  treatment  effect. 

gl^fadx^n  weight  listed  for  Math  117  indicates  a  size- 

f  ■  k\-     "      ,     y  \  v*  <  ■■,     "  •  •  ■  '  » 

able  point  difference  between  conditions,  6.14  points  or  about  three-fourths 
of  a  standard  deviation.    However,  there* is  an  interaction  term  in  this 
regression  model.  .  Thy  actual  estimated .  effect  in  ttje  day  division  would  be  * 
a  combination  of  terms:  *  \ 

•  x    •-  6.14  fpr  6tudentsN;dmpifeting  an  evening  TlCCIT  class  and 

Tn'«-  6.14.  +  (-7.88)  -    -1.74- for  students  completing  a  day 

TICCIT  class.  ,  :y  V 

.' ■    (r-      :.  ■       f   :■■ :  ■    •  •  • 

The  significant  effect  of  6.14:.appl-±ed,  to  the  evening  division  alone,  and  just 

-  *  .  "    •    „  ■■■  ■         v  ,  i    '  •  . 


four  students  of  the  20  observation^  in  the  TICCIT  condition  represent  the 

.  ■  :,-•  ;>,  - 

relevant;  population.,  The  small,  sample  elze  may  therefore  raise  doubt  about 
the^lk«ufts>  for  .Math  117.  .* .  ' 

At  Alexandria  it  ift  possible  that  atudents  completing  a  math  course  under 

■ .  ■      ■     v     T     :   . : 

the  TICCIT  program  within-  oW  academic  ,t*rm.  Would  demonstrate  the  program  s 


effectiveness.,  j Ther>  dete  Ju^t  33  sucfe  students  In  the  winter  quarter  with 
>oth  a  protest  and  a  posttest,  far  Math  31  (13  for 'the  TIGCIT  conditioji,  ll  for 
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Achievement  Results:  Analysis  on^onmon  Objectives 
Math  Courses  at  Phoenix 
.   Spring  and  Summer  Semesters,  1976 


i 

Course 


Number  of  Students  - 
Multiple  Correlation 


Math  106 
N  ■  101 
R-  .67 


Standatd<'Erroi  of  Estimate     se  =5.77' 

■   y     •  ,  est 

Regression  Value  of 
.Independent  Variables:      Weight  t-statistic 

Condition:  .  ,« 

TICCIT(l)/Lecture]CO) 


Entrance  Ability: 
Pretest * 


V 


Division: 
Day(l)/Evenihg(0) 

Term: 

>  Spring  (1) /Summerf)) 

Interactions: 
Day  &  Spring 
TICCIT*  Day 

Constant: 


3.37 


1.44 


1.49 


*P  <  .05 
***p  <  ,001 


2.07* 


7.64*** 


1.21 


Math.  108 

N  B '56., 

R  «  .30, 

se  =7,33 
est 


Regression  Value{  of 
Weight,  t-statistic 


.97 


31.31 


.43 


46  1.56 


-4.62  -1.51 


t.     Math  117  * 

■   N  -  99 

R  -  .71 

se  =5.33 
est 


Regression  Value  of 
Weight  ^-statistic 

f 

6.14  2.05* 
.66  3.93*** 


•1.22 


-.62 


-11.16  -5.79*** 


-11.18  4.57*** 
-7.88  -2.36* 

17.46 
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lecture  and  9  for  programmed  instruction) .    The  analysis  of  these  data  would  ' 
indicate  a  significant,  positive  treatment  effect:     t  -  A. 60  and  t  -  2.19, 
j>  <  .05,  for  a  comparison  of  tfce  TICCIT  and  lecture  conditions;  t  ■  3.66  and 
t  ■  2.19,  j>  <   .05,  for  a  comparison  of  the  TICCIT  and  programmed  conditions. 
The  small  numbers  cf  observations  might  once  again  tempt  us  to  dismiss  these 
results.    To  do  so  would  be  to  disregard  a  significant  effect  ftor  the  third 
time,  counting  Math  32  in  the  winter  and  spring  quarters,  Math  106  in  the 
extended  spring  semester,  and  now  Math  31  in  the  winter  quarter.    The  sample 
size  may  indicate  that  our  estimated  effects  apply  to  a  select  group  of 
students,  but  nevertheless  they  constitute  evidence  of  TICCIT' s  impact  on 
student  achievement. 

It  appears  that  when  we  remove  the  time  constraints  of  an  academic  term, 

the  sample  size  Increases  and  the  variable  representing  the  instructional 

t 

"condition  is  more  apt  to  become  a  significant  term  in  the  regression  model. 

>' '.' 

This  happened  for  Math  32  (Table  5.6),  Math  106,  and  Math  117   (Table  5.7)  when 
posttest  data  were  taken  from  multiple  terms.    Table  5.8  presents  analyses 
for  the  introductory  algebra  courses,  Math  007  and  Math  31,  across  academic 
terms.    With  relaxed  time  constraints  a  positive  TICCIT  effect  becomes  apparent 
in  both  courses.     At  Phoenix  the  effect  was  5.40  points  on  the  posttest  score 
for  common  objectives.    This  difference  corresponds  to  approximately  two-thirds 
of  a  st^i^ard^eviatiofi  or  better  than  ten  percent  of  the  total  score  possible. 
It  applies  only  to  the  evening  division,  as  indicated  by  the  treatment-division 
interaction  term  in  the  regression  model  and  suggested  by  the  summary  of  achieve- 
ment data  (see  Table  5.1),    At  Alexandria  the  effect  in  the  day  division  was 
4.39  for  a  comparison  of  TICCIT  results  with  those  of  lecture  classes.    Such  a 

point  spread  between  conditions  is  equivalent  to  ten  percent  of  the  total  score 

' ....  .  .  ...... 
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Achltvntyt  Risulti  acrosi  Teru:  Afiilyili «  Coaon  Objectives 
Alexudria  and  Phoenix 
Math  31  aad  Math  007 
Acad«ic  Year  197J-76 


Number  of  Students 
Multiole  Correlation 


Math  007 
N  -  331 
R  -  .65 


Math  31 

Students  with  Credit 
N  *  87  ■ 
R-  .64 


Math  31 

Students  with  Credit 
H  "  110 
R-  .58 


Independent  Variables : 

Condition: 
TICCIT(1)/Lecture(0) 
TICCIT(l)  /Progrmd(-l) 
Lecture  (1) /Programed  (-1) 

Entrance  ability: 
Pretest' 

Acadocic  Tern: 
Fali(l)/3pring(0) 
Fall(l)/Sprlng(-1)  . 
Hinter(l)/Spring(-1) 

Division: 
Day(l)/Evening(0) 

Student  Status: 
Full-tiafe (1) /Part-timt (0) 

Employment: 
Full- 1  i  ne (3 ) /Par t- tine ( 2 ) /None (1 ) 

Enrollment  in  Course; 
Prior  enrolltnent(l)/|irst  enrolbenc(O) 

Maturity: 
Age  (Years) 

Interactions: 
TICCl':'*Day 

Constant: 


Regression 
Weight 


5.40 


.45 
-1,36 


3.62 


-5.47 
14.12 


Value  of  Regression  Value  of  Regression  Value  of 
t-statistic    Weight      t-statlstlc    Height  t-statistlc 


2.69 


14.53 


-2.00 


*** 


4.67 


*** 


-2.45 


-.11 


.21 


.68 
.10 


-3.16 


-1.40 


.28 


-1.83 


.14 


4,50. 


32.48 


-.06 


4.04 


.93 
.12 


-2.07 


-1.17 


-  .40 


-1.45 


-1.73 


2.12 


*** 


1.33 
-1.08 


.17 


.44 

.32 


-  .97 


-1.21 


.89 


.03 


1.93 
-1.78 


4.03 


.67 
,52 


.79 


-1.10 


-1.39 


*** 


-2.35     |  -2.22* 


-  .43 


28.65 


*** 


P  <  .001 

ERIC  / <  '01 

TO3  p'-05 


Table  5.8 
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possible  on  common  objectives.     The,  estimated  effect  was  less  for  TICCIT 
results  compared  with  those  of  programmed  instruction.     Then  the  regression 
coefficient  was  1.33,  which  corresponds  to  a  t  of  2.66  points  and  approaches 
accepted  limits  for  statistical  significance.     The  lower  effect  for  TICCIT 
when  compared  against  programmed  instruction  should  be  expected  since  there 
were  fewer  posttest  items  common  to  these  two  conditions  (38  rather  than  44) 
and  this  estimate,  unlike  that  obtained  with  lecture  as  a  relative  standard, 
he^d  for  day  and  evening  classes. 

Students  able  to  complete  a  math  course  on  th<-  TICCIT  system  do  attain 
higher  posttest  scores  than  comparable  students  from  lecture  classes.  These 
treatment  effects  on  student  achievement  may  not  be  readily  apparent  from 
analyses  conducted  for  each  academic  term.     However,   the  TICCIT  program  offers 
students  self-paced  instruction.     This  both  removes  much  of  the  need  for 
students  to  meet  a  fixed  time  constraint,  and  introduces  a  temptation  for 
them  to  neglect  a  TICCIT  class.     Especially  toward  the  end  of  an  academic 
term  when  the  demands  of  other  courses  must  be  met,   students  may  choose  to 
take  advantage  of  TICCIT's  self-paced  featutu  and  plan  to  continue  studying  on 
the  TICCIT  system  after  the  forq^I   close  of  a  term.     So  analyses  of  achievement 
data  collected  within  an  academic   term  may  not  reveal  treatment  effects.  But 
analyses  across  academic   terms  show  a  significant,   positive  treatment  effect 

in  five  math  courses  of  six. 

»>£ 

Ability  and  Content  Subscores.     Regardless  of    its  overall   impact  on 
student  achievement,   the  T1CCII  program  might  be  particularly  strong  or  weak 
for  certain  components  ot   the  math  curriculum.     Such  a  view  is  consistent  with 
the  belief  that  competing  currlcular  programs  may  lead  to  different  patterns 

ERIC  3 13 


-  261  - 

o  • 

measurement  within  one  comparison  group).     Even  further,  data  taken  from  a  few 
academic  terms. for  several  courses  may  reveal  an  effect  peculiar  to  the  con- 
'ditions  of  one  specific  term  or  the  materials  appropriate  to  one  particular 
course.     All  j^f  these  considerations  enter  into  our  analyses  for  student 
achievement  so  that  we  might  isolate  and-estimate  TICCIT  effects, 
\ 

5 •  3 •  a    Math  Courses  ' 

The  posttests  prepared  as  measures  of  student  achievement  in  math 
courses  actually  gave  a  number  of  different  subscores.     The  basis  for  our  com- 
parisons of  TICCIT  results  with  those  of  regular  classroom  practices  was  the 
subscore  that  reflected  course  objectives  shared  by  both  conditions.  This 
subscore  for' common  objectives  typically  included  about  85  percent  of  the 
items  on  a  course's  posttest.     The  other  items  concerned  objectives  unique  • 
to  either  the  TICCIT  program  or  lecture  classes,  and  often  two  or  three 
questions  went  beyond  the  course's  objectives  regardless  of  condition.  The 
analyses  presented  here  deal  first  with  common  objectives  and  then  with 
objectives  unique  th  one  condition, 

It  may  be  recalled, that  our  approach  to  data  analysis  involved  estimating 
treatment  effects  for  similar  students.     Similarity  was  to  be  defined  by  a 
number  of  demographic  variables,  such  as  a  student's  entrance  ability  in  the 
subject  matter  and  age.     Requiring  these  additional  data  meant  the  loss  of 
some  observations  in  the  analysis  due  to  incomplete  information.     This  happened 
in  the  analyses  for  course  completion,  but  the  number  of  observations  was  so 
large  that  we  could  afford  the  loss,*    With  the  number  of  pretest-posttest 
observations  already  low  for  math  courses  under  the  TICCIT  program,  we  had 
to  retain  all  of  the  observations  and  chose  not  to  sacrifice  any  for  missing 
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of  test  performance  rather  than  any  difference  in  aggregate  achievement  (see 

Walker  &  Schaf f arzick,  1974).    Certainly  if  test  coverage  favors  one  curriculum 

or  another,  there  should  be  achievement  differences* reflecting  that  bias.  But 

if  we  restrict  comparisons  to  just  those  test  items  reflecting  objectives  shared 

by  the  ^ompeting  curricula,  we  can  eliminate  possible  sources  of  test  bias.  Then 

comparisons  of  competing  curricular  programs  would  yield  fair  estimates  of  their 

relative  advantages  for  specific  outcomes.    All  of  our  comparisons  for  detecting 

relative  advantages  on  specific  achievement  outcomes  draw  on  course  objectives 
* 

common  to  the  T ICC IT  program  and  the  conventional  math  program. 

The  subscores  formed  from  common  post test  items  reflect  two  dimensions  of 
the  math  ci^^culum.    One  series  of  subscores  divides  test  items  into  four 
separate  ability  levels^     (0)  factual  recall;   (1)  manipulation;  (2)  solution  of 
routine  problems;  and  (3)  demonstration  of  concept  comprehension.    The  order  of 
these  four  ability  levels  constitutes  a  progression  from  cognitive  tasks  requiring 
just  knowledge,  to  others  demanding  comprehension,  to  those  calling  for  the 
straightforward  application  of  rules,  and  finally  to  tasks  necessitating  the 
analysis  of  concept  relationships  in  unfamiliar  problem  situations  (see  Bloom, 
1956).    Another  dimension  corresponds  to  the  content  categories  within  a 
test:     (1)  arithmetic;  (2. a)  simplifying  signed  and  complex  numbers;   (2.b)  equa- 
tions;  (2.c)  exponents,  radicals  and  logarithms;  (2.d)  factoring,  multiplying 
and  dividing  algebraic  expressions;  (2.e)  number  line  and  coordinate  plane; 
(2.f)  inequalities,  absolute  values  and  sets;  (2.g)  sequences,  arithmetic  and 
geometric  progressions;  and  (2.h)  permutations  and  combinations,  binomial  theorem. 
These  content  categories  represent  a  classification  of  the  complete  precalculus 
curriculum.    Not  every  category  appears  on  each  posttest.    The  posttests  for 
beginning  algebra  do  not  cover  progressions  or  permutations  and  combinations,  , 
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while  those  for  intermediate  algebra  oMlt  Items  on  arithmetic.    The  actual  number 
of  items  within  each  ability  level  or  content  category  probably  gives  a  strong 
indication  of  course  emphasis. 

Naturally  the  common  objectives  broken  down  further  to  ability  or  content  , 
subscores  leave  few  items  to  define  a  specific  outcome y  Therefore,  our  measure- 
ment of  specific  outcomes  cannot  be  as  reliable  as  the  total  score  on  common 
objectives.    Table  5.9  presents  reliability  estimates  for  posttest  subscores. 
Although  some  approach  an  acceptable  reliability  for  group  measures  (i.e., 
•80/,  most  seem  too  unreliable  for  firm  conclusions  about  specific  program 
outcomes.    The  results  of  data  analyses  for  subscores  do' provide  descriptive 
evidence  that  should  be  useful  in  planning supplications  of  the  TICCIT  program 
or  developing  computer-assisted  instructional  systems. 

Appendix  P  contains  summaries  of  student  performance  on  the  math  posttests. 
The  mean  and  standard  deviation  appear  for  each  subscore  along  with  the  number 
of  items  in  that  subscore.    From  these  data  we  tdok  subscores  on  ability  levels 
and  content  categories.    Each  score  became  the  dapfendent  variable  in  a  regression 
analyses  similar  to  those  shown  in  Tables  5.5  and These  were  fallible 
dependent  variables  in  that  they  often  included  as  much  variation  due  to  error 
as  due  to  true  score.    Interpretation  of  subscore  results  was  further  compli- 
cated by  the  large  number  of  analyses.    Since  a  course's  posttest  had  subscores 
for  four  ability  levels  and  six  or  more  content  categories,  there  were  at  least 
ten  analyses  per  course. 

Table  5.10  presents  the  results  of  these  analyses  in  the  courses  for  which 
subscore  reliabilities  were  given  by  an  earlier  table  (see  Table  5.9).  It 
includes  the  regression  coefficient  and  t-statistic  for  tY\e  independent  Variable 
that  represented  the ^instructional  condition  in  each  analysis.    As  before,  the 


Poituit  Subscors  Iiliahilityi  Iatinial  Cooilitwy  Utimtu 
Alexandria  and  Phoeni* 

Mathematics  Courses 
Academic  Year  1975-76 


Objectives 

,.,    1  Unique  to  lecture 

3  Unique  to  TICCIT 

<t  Beyond  curriculum 


Number 
of  Icettj 

5 

,4 
0 


Math  007 

Fall  Spring 
(N  -  132)  (H  -181) 


r 

.67 


r 

.61 


Number 
of  Items 

1 
9 

0 


Math  31 


Hath  117 


,2 


Fall  Winter  Spring  1  Spring 
(N  -  58)    (N  -  80)    (K  -  79)  (N  -  71) 


.51 


.45 


Number 
f  Items 


.69 


Math  32 

Winter  and  Spring 
(N  ■  92) 

Number 

of  Items  r 

0  . 

4 

3 


Ability  Levels  - 

0  Factual  Recall 

1  tonipulation 

2  butiuc  Problem 

Sclviag 

3  Concept 

Comprehension 

Content 

1  ^ithnctic 

2  Algebra 

a  simplifying 
b  equations 
c  exponents , 

radicals 
d  algebraic 

expressions 
e  nuttier  line  and 

coordinate  plane 
f  inequalities, 

absolute  value 
g  sequences  and 

progressions 
%  h  permutations  and 

combinations 


2  , 
19  (18)1 

22(17) 

.81 

.79 

.69 
.73 

2 

15  1 
22 (18)1 

.54 
.66 

.82 
.84 

.83 
.86 

1 
5 

16(11) 

.46 
.75 

1 
3 

18(16) 

.69 

14(11) 

.63 

.48 

15(9) 

.45 

.65 

.76 

24(21) 

.82 

19(14) 

.74 

12(10) 

.64 

.47 

12(H) 

.56 

.69. 

.  .75 

0 

0 

6(5) 
18(17) 
5 

:66 
.83 
.47 

.49 
.76 
.40 

7 

15(14) 
7(4) 

.40 
.63 

.75 
.82 

.73 
.85 

3 

16 

3(1) 

.75 

3(1) 
12(10) 
14(13) 

.71 
.65 

10(7) 

.70 

.64 

4 

4(3) 

3 

3 

5(3) 

12(11) 

.77 

5(4) 

3(1) 

4(1) 

1 

4(3) 

'4(2) 
3(1) 


*The  number  of  items  in  parentheses  reflects  only  those  items  on  common  objectives.  Responses  to  these  common  items  were  the  basis  fo^ 
reliability  estimates,  W 

2 

In  the  fall  quarter,  the  reliability  estimates  vera  taken  from  data  on  those  students  who  received  course  credit. 
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A-lex^dria  and  Phoenix 

Math  Courses 
Academic  \'tar  1975-76 
Xath  117 


Math  007 
Spring  Smarter 
N  ■  172  i 


Suiuur  Semester 
N  -  99 


Hath  31 
Fall  Quarter 


Math  31 
Winter  Quarter 
'  N  -  36 


Regression   Value  of     Rcgrsssion   Value  of     Regression   Value  of     Regression   Value  of 
Weight    ^statistic    '  Weight    t-atatlstic     Weight    t-statlatic     Weight  Mjtatlstlc 


Math  32 
Winter  4  Spring  Quartan 
N  -  44 

Regrtssioi  Value  of 
Weight  j-statistlc 


.10 
.63 

1.75 
.38 


Content  Categories: 

Arithaerfc  k  .07 
algebre- 

Sipliiy.^ 

EqLuticns  2.53. 

Exports,  Radicals,  j 
Logtfithas  .13 

Factoring,  Multiplying, 
IWAiti  Expressions  .20 


Nufiber  Line  and 
Ccordir.au  Plana 


.17 


Inequalities ,  Absolute. 
Values  -.01 

Sequence,  Progression 

PenuMtioas, 
CocMoatiots 


1.16 
1.05 

2.84** 


.21 

-.91 

1.S4***' 

.47 

.53 
1.C2 
-.0? 


.10 
.13 

.11 
.01 


.00 

-.10 
1.26 

-.55 

-.28 

.15/ 

J 


.1.21 
.25 

.20 
.01 


.00 

-.45 
2,20* 

-2,34* 


.91 
-1.27 


-.24 
-.04 

.00 
.44 


-.23 
.51 

.17 

-.42 

-.43 

.12 
.24 
.20 


-1.85 
-.17 

.00 
.45 


-1.09 
.71 

1.60 

-1.90 

-.77 

1.00 
1.31 
1.63 


.10 
.K 

1.25 
.29 


.44 

.06 
.76 

.65 
-.12 

.06 
-.07 


.44 

I 

.21 

1.25 
.59 


.16 
1.01 

1.34 

-.32 

.22 

-.38 


.01 
1.38 


2.60* 

79  .81 
72  1.48 

( 


.93 

.34 
.66 

.90 
-.11 

.20 
-.05 


2.26* 

1.21' 
.96 

2.00 

-.24 

.72 

-.31 


.33 
-.17 

2.57 


"  1.58 
-.42 

2.62* 
1.37 


1. 

1.69* 
.49 
.14 


.35 
1.67 

2.43* 

1.47 

.29 

.99  • 


04 


*p  <  .05 
**?  <  .01 
***p  <  ,001 
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coefficients  correspond  to  the  estimated  point  difference  between  outcomes 
of  the  TICCIT  program  and  those  of  the  lecture  condition.    Perhaps  the  most 
striking  feature  of  these  results  is  the  absence  of  significant  treatment 
effects.    With  so  large  a  number  of  analyses  we  might  expect  to  find  more 
than  eight  significant  effects  for  specific  outcomes.    And  the  significant 
treatment  effects  on  test  performance  taken  as  total  score  on  common  objec- 
tives  would  reinforce  such  an  expectation.    But  three  facts  should  reduce 
/f'the  number  of  significant  findings:     the  elimination  of  possible  sources  of 
test  bias  by  considering  only  items  on  ^common  objectives;  the  fallible 
nature  of  subscores  as  dependent  variables;  and  the  breakdown  of  a  possibly 
significant  overall  effect  into  several  smaller  parts. 

Only  two  significant  effects  were  repeated  in  another  math  course  or 
"academic  term.    One  corresponds  to  the  solution  of  routine  problems  as  an 
ability  level  and  the  other  to  equations  as  a  content  category.    Both  of  these 
specific  outcomes  involved'an  effect  in  favor  of  the  TICCIT  program.  The 
point  differences  reflected  by  the  regression  coefficients  were  always  posi- 
tive for  these  two  subscores,  and  they  were  significant  in  two  cases  for  each 
outcome  (Math  007  in  the  fall  semester  and  Math  32  in  the  winter  and  spring 
quarters  for  the  solution  of  routine  problems,  and  Math  007  in  the  fall 
semester  and  again  in  the  spring  semester  for  equations)^  Further,  the 
Estimated  treatment  effects  for  Math  106  in  the  fall  semester  were  positive 

add  significant  for  both  the  solution  of  routine  problems,  t  *  2.29  and  t  m 

•  \  ' *    '  :  t  . 
2;86j  j>  <  .01,  and  equations,  t  -  1.13  and  t  -  2.72,  £  <  .01,  as  specific 

outcomes.  1 

*■ .  * 

^The  probable  reasons  for  these  two  specific  strengths  in  the  TICCIT 
program  come  from  a  fundamental  feature  of  its  design  and  the  detail  of  its 
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content  coverage.    The  computer  system  provides  students  with  access  to  %an 
extensive  file  of  practice  exercises  for  learning  and  repeating  rules  of 
problem  solving.    Students  take  advantage  of  this  opportunity  and  spend  most 
of  their  time  on  the  T1CC1T  system  doing  practice  problems  (see  Chapter  7:  \f 
Student  Activities).    One  result  of  their  repetitious  practice  in  applying 


rules  is  improved  performance  in  solving  routine  problems.    The  advantage ^  J^^ 
for  the  TICCIT  program  in  posttest  subscores  related  to  equations  may  be  £ 
attributed  to  the  depth  of  its  coverage  on  this  topic  (see  Appendix  A,  in 
particular  Unit  20).     It  seems  for  the  topic  of  equations  the  developers 
succeeded  in  isolating  the  key  attributes  of  rules  and  providing  appropriate 
instances  of  r?ule  applications  as  examples  and  practice  problems. 

A  final  pol,nt  about  the  ability  and  content  subscores  should  be  noted. 

It  concerns  the  absence  of  significant  effects.    The  TICCIT  program  resulted 

v       ■  s 

in  test  performance  comparable  to  lecture  results  for  the  highest  abili 


level  and  most  advanced  content  category.    The  direction  of  differences  on 
these  specific  outcomes  shoved  consistently  positive  TICCIT  effects.  The 
program's  impact  on  student  achievement  was  not  just  a  matter  of  rote  factual 
recall  or  simple  arithmetic  operations.     It  affected  student  achievement  in 
substantial  and  substantive  parts  of         math  curriculum:     the  solution  of 
routine  problems  and  equations.    The  TICCIT  program  also  held  £ts  own  on 
specific  outcomes  for  which  conventional  practices  might  be  thought  to  be 
more  conducive  to  student  learning. 

Unique  Object ives.    Few  of  the  posttest  items  represented  objectives 
unique  to  either  the  TICCIT*  program  or  lecture  instruction.    There  was  sub- 
stantial overlap  in  course  objectives  across  conditions  since  all  classes 
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had  to  meet  the  same  curricular  requirements.  However,  one-sixth  of  the  posttest 
Items  for  Algebra  1  represented  material  taught  only  in  T1CC1T  classes.  This  is  a 
reflection  on  the  ambitious  cova*^  of1  that  course  under  the  TICCIT  program  (see 
Figure  2.1);  the  posttest  for  Math  31  merely  makes  the  discrepancy  in  coverage 

explicit.    Usually,  fewer  than  ten  posttest  items  corresponded  to  unique  objec- 

tj 

tives  (see  Appendix  C  and  Table  5.9),  and  these  were  split  among  objectives 
unique  to  the  TICCIT  condition  and  objectives  unique  to  the  lecture  condition. 

Table  5.11  presents  the  results  of  the  analyses  for  posttest  subscores  on 
unique  objectives.    The  estimated  treatment  effects  here  are  subject  to  the  same 
qualification  that  applied  to  ability  and  content  subscores:     the  dependent 
variable  is  a  fallible  measure  of  specific  program  outcomes.     But  the  statistical 
tests  of  significance  suggest  a  clear  pattern  of  the  results.     Students  taking  a 
mattv  course  on  the  TICCIT  system  did  attain  higher  posttest  subscores  on  objec- 
tives unique  to  the  T-ICCIT  program,  while  students  in  lecture  classes  attained 
higher  posttest  subscores  t>n  objectives  unique  to  the  lecture  condition.  Had 
such  a  pattern  not  been  obvipus,  there  would  be  reason  for  doubting  the  classi- 
fication of  posttest  items  by  objectives.     Instead  it  seems  the  procedures  for 
classifying  items  \forke\Jell  arid,  hence,  the  comparisons  of  program  results 
according  to  common  objectives  were  fair. 

The  estimated  treatment  effects  given  in  Table  5.11  also  suggest  that  pro- 
gram comparisons  based  on  total  posttest  scores  would  lead  to  greater  observed 
differences  in  test  performance.    At  least  this  would  be  so  for  Math  31.  For 
that  course,  including  items  on  unique  objectives  might  increase  the  overall 
point  difference  between  conditions.    Another  point  added  to  the  TICCIT  effects 
already  estimated  |or  Math  31  (see  Table  5.8)  would  provide  further  evidence  of 
a  positive  TIC<^|  impact  on  student  achievement. 
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Achievement  Results:  Analysis  on  Unique  Objectives 
Alexandria  and  Phoenix 

Hath  Courses 
Academic  Year  1975-76 


■»  Dependent  Variable 


Posttest  Items 
Unique  to  Lecture 


Posttest  Items 
Unique  to  TICCIT 


Course 


Term 


Number  of  Regression 
Students     Weight  (t) 


'  Value  of 
t-statistic 


*** 
** 


Math  007 

$Fall  Semester 

152 

-1.73 

-5.80 

Math  007 

Spring  Semester 

172 

-  .85 

'  -2.971 

Math  117 

Spring  &  Summer 
Semesters 

99 

.09 

.48 

Math  31 

Fall  Quarter 

48 

-  .02 

-  .11 

Math  31 

Winter  Quarter 

36 

.19 

1.33 

Math  32 

Winter  &  Spring 
Quarters 

44 

Regression 
Weight  (t) 


1.51 
1.09' 
.30 


Value  of 
tt-statlstlc 

*** 


.91 

4.04 

*** 

'i 

.86 

1  4.24 

00 

0 

,49 

2.53 

1 

3.02 
1.97 
.82 


** 


\ 


p  <  .05 


** 


p  <  .01 


*** 


p  <  .001 
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5.3.b    English  Courses 

The  posttests  prepared  as  measures  of  student  achievement  in  English 
courses  had  two  distinct  components.    There  was  an  objective  test  of  writing 
skills 'that  covered  traditional  points  in  composition  (see  Apf^ndix  It   *  • 

presented  questions  about  grammatical  and  structural  conventions  in  standard 
written  English  and  called  for  responses  in  a  multiple-choice  format  (see 
Figure  2.5).     Obviously  such  a  test,  as  an  indirect  measure  of  writing  ability 
and  a  sample  of  pertinent  skills,  had  limitations.     The  English  posttests, 
therefore,  included  an  essay  component  (see  A|^fidW  b)  •     Students  wrote  on  a 
given  topic  under  test  conditions,  and  instructors  later  assigned  scores^to 
these  essays  in  grading  sessions  supervised  by  the  evaluator.    A  student's 
scores  on  the  objectives  and  essay  components  served  as  complementary  but 
separate  measures  of  achievement. 

For  English  courses  the  question  of  program  impact  on  student  achievement 
became  a  matter  of  point  differences  on  either  the  essays  or  objective  test. 
Analyses  again  took  the  form  of  regression  models.    A  component  of  the  English 
posttest  was  the  dependent  variable,  and  the  independent  variables  defining 
similar  students  in  each  condition  were  the  same  'as  those  employed  in  the 
analyses  for  course  completion.    The  estimates  of  treatment  effects,  then, 
represent  the  difference  in  scores  between  students  completing  an  English 
course  on  the  TICCIT  system  and  comparable  students  from  lecture-discussion 
classes. 

Tables  5.12  and  5.13  presenfc^the  analyses  for  student  achievement  as  a 
program  outcome.     It  may  seem  as  though  the  estimated  treatment  effect  was 
negative  as  often  a?  positive.    The  regression  coefficients  for  the  term 
representing  the  instructional  conditions  were  negative  in  six  of  ten  cases. 
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M      "  -37 
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Ohjtctivt 
Fall  S«a».c«r 
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ll4^D*notnc  Variablss: 

fti|TOtttlOtl 

want* 

Vslus  ot 

t-SUtlittC 

tafrMtlaa 

i    Valut  of 
t-ttttittU 

Vtlut  of  . 
t-ttttlttlfl 

Itfrtttlm 
Wtltht 

Vtlut  tf 
t'limlHI 

Condition; 

nCCIT(l)/Ucturt(0) 

.204 

1.42 

-3.7/ 

— x.  n 

-.498 

-3.03** 

1.68 

2.41* 

Intraac*  Ability: 
ItMy  Prstatt 
Objtctlv*  Pretsat 

.318 
.035 

3.19** 

-.45 

3.06** 
-1.07 

.279 
.036 

4.56*** 
5.65*** 

1.01 
.44 

2.06* 
8.59*** 

Division: 

D*y(l)/lv«rving(0) 

.341 

1.S7 

-1.42 

-1.20 

-.267 

-1.81 

1.12 

1.05 

Instructor: 
il 
12 

-.284 

-1.26 

-.31 

-.21 

-.234 
.082 
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.64, 
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Itudtnt  sti-us; 
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1 
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-J 
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-.12 
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taployawnt : 
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-.29 
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.11 

2.14* 
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*** 


p<.05 
*p<-01 
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All  (Tun) 

Ur. 

felt(l)/fuile{0) 
Iatitictloo: 
TICCIt*  Objective  Freest 

nan*  nej 

Cooitent- 


.470  1.63 

.209  1.76 

.019  1.58 

-.151  -.91 

-.617 


P<.05 


M 


p<.01 

It 

P<.001 


-.93 

3.36 
,42 

1.03 

-.91 


5.20*** 
4.56*** 

.43 
-.58 


-1.57  -1.14 

-.88  .55 

-.83  -1.22 

-.13  -1.83 

2.62  2.74*' 

14.03 


-.164  -1.17 

•202  2.71** 
.043'  6.04*** 

•141  .94 


-.059 


■■50 


•189  .85 


-.021 
-.001 


-.33 
-.17 


-.068  -.70 


1.03 


■'•"l 

1,09 
.54 

-1.26 

-2.25 


.72 


-.51 
'  -.01 
-1.55 
.26 
15.39 


-3.09** 

1.93 
7.97*** 

-1.06 

-2,63** 


.79 


LSI'  1,11 


-1.00 

-.21 
-2.09* 

2.75** 


'  Eisay 

Ml  qurter 


eit 


.149 
.105 


.226  1.43 

.199  l77 

-.003  -.03 

•OH  1.34 

-.007  ..07 

■.560  -2.25* 
.383 


Objective 
fall  Quarter 


158 

? ' 

167 

,55 

s  * 

.75 

•  .65 

la 

est 

•4.25 

value  of 
Mtatlitle 

aegreseloa 

witht 

Valutof 
t-atatlitlc 

2.11* 

-.90 

-1.14 

1.40 
4.67*** 

.89 
.57 

1.7* 

10.41*** 

1.49 

.45 

.42 

-1.05 

.72 
.52 
1.75 

-1.94 
1.20 
.07 
'  .28 

-1.57 

.05  1 
.25  f» 

2.23  2.25* 


• 

U) 


.64 


.38 


.19  .35 

.01  .20 

1.03  1.54 

9.69 
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And  the  statistical  tests  associated  with  the  condition  variable  were  signifi- 
cant four  times,  twice  with  positive  weights  and  twice  with  negative.  But 
these  regression  coefficients  did  not  necessarily  reflect  the  treatment  effect. 
Whenever  an  interaction  term  involving  the  condition  variable  accounted  for  a 
significant  portion  of  posttest  variance,  it  entered  into  the  regression  model. 
Then  estimating  treatment  effects  also  meant  combining  regression  weights. 

The  first  analysis,  including  an  interaction  term  in  the  regression  model, 
is  that  jor  the  objective  *est  of  writing  skills  in  the  spring  semester  at 
Phoenix.    Actually,  the  response  functions  for  student  achievement  in  this  case 
were  neither  linear  nor  parallel,    the  addition  of  the  squared  score  on  the 
objective  pretest  as  an  independent  variable  indicates  nonlinearity .  The 
treatment-trait  interaction  term  indicates  a  relationship  between  variables 
in  one  condition  but  not  the  other.     Here  there  was  a  relationship  between  a 
student's  score  on  the  objective  posttest  and  his  age  that  held  just  within 
the  TICC1T  condition.    Put  another  way,  TICCIT's  impact  on  student  achievement 
varied  according  to  age. 

Our  best  estimate  of  a  treatment  effect  in  cases  with  trait- treatment 
interactions  is  the  value  of  the  regression  coefficient  for  the  condition  term 
dIus  the  value  of  the  regression  coefficient  for  the  interaction  multiplied 
by  the  trait's  mean  value  within  t£  TICCIT  condition.     It  is  much  easier  to 
understand  this  combination  of  coefficients  as  it  applies  to  the  achievement 
data  from  the  spring  semester.    The  estimated  treatment  effect  would  be: 
x  -  -3.77  +  .23(26*6) 
t  -  2.38.  f 

It  is  the  weight  for  the  condition  term,  -3.77,  plus  the  coefficient  for  the 
trait-treatment  interaction,  .23,  multiplied  by  the  average  age  of  students 
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within  the  TICCIT  condition,  26.46.    Since  the  population  referred  to  by  our 
estimated  treatment  effects  is  always  that  which  completed  a  course  on  the  TICCIT 
system,  the  mean  of  a  trait  must  be  taken  from  observations  for  that  population. 
Thus,  26.46  was  the  mean  age  of  students  completing  English  019  (or  English  29 
in  the  evening  division)  on  the  TICCIT  system  in  the  spring  semester. 

The  estimated  treatment  effect  of  2.38  points  on  the  objective  posttest 
is  significant  (t  -  2.48,  R  <  .05)  but,  in  a  sense,  incomplete.    The  point 
difference  varied  according  to  age,  and  estimates  made  at  different  intervals 
in  the  age  distribution  would  give  a  fuller  picture  of  effects.    At  approxi- 
mately one  standard  deviation  below  and  above  the  mean  age  of  TICCIT  students, 
the  treatment  effects  would  be  .42  for  students  18  years  of  age  and  4.37  for 
students  35  years  old.     The  impact  of  the  TICCIT  program  on  objective  test 
performance  was  consistently  positive  at  all  age  levels  and  apparently  increased 
with  age. 

The  TICCIT  program  had  a.  positive  effect  on  student  achievement  across 
semesters  at  Phoenix.     In  the  spring  semester  that  effect  amounted  to  2.38 
points  on  the  objective  test  of  writing  skills.    The  same  significant,  positive 
effect  occurred  in  the  fall  semester.     Then  the  poinc1  difference  in  favor  of 
the  TICCIT  program  was  1.68  in  terms  of  score  on  the  objective  posttest.  This 
estimated  treatment  effect  was  independent  of  a  student's  age,  but  age  still 
served  as  a  significant  independent  variable  by  itself,  b  -  .11  and  t  -  2.14, 
£  <  .05.    Descriptive  achievement  data  from  the  summer  semester  (see  Appendix 
0)  provide  supporting  evident  for  a  positive  treatment  effect  on  objective 
test  performance.     But  perhaps  this  occurred  at  the  expense  of  students'  actual 
writing.     Discrete  skills  might  be  learned  through  extensive  practice  but  not 
applied  in  essays. 
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The  estimated  treatment  effects  on  the  essay  posttest  seem  to  depict  a 

mixed  result  for  the  TICCIT  program.    The  regression  coefficient  for  the  term 

representing  the  instructional  conditions  was  positive  in  the  spring  semester 

and  highly  negative  in  the  fall.    However,  there  should  be  two  estimates  of 

treatment  effects  for  the  fall  semester.    The  estimate,  for  the  day  division 
*  • 
would  be  a  combination  of  coefficients,  (-.498)  +  (.581),  yielding  *(  slight 

•  •  it 

positive  value,  td  -  .08.    Another  for  the  evening  division  woulji|j|p:he 
coefficient  for  the  condition  term  as  it  stands,  t  -  -.498  and  t  w  ,-3.03, 
<  .01.    The  probable  reason  for  this  negative  effect  in  English  029  rests  in 
the  context  for  the  demonstration.    Part-time  instructors  unfamiliar  with  the 
TICCIT  program  taught  evening  classes  on  the  computer  system  during  the  fall 
semester,  and  their  inexperience  showed  in  the  achievement  results.  This 
suggests  that  instructors  must  be  thoroughly  familiar  with  the  TICCIT  program 
in  order  to  teach  with  it  in  an  effective  manner. 

At  Alexandria  there  were  two  instances  of  a  significant  treatment  effect. 
The  first  involved  student  performance  on  the  objective  posttest  in  the  spring 
quarter.    Estimating  that  ^ffect  as  a  combination  of  regression  coefficients, 

\  -  -7.79  +  .26(21.48) 

t  -  -2.21, 

indicates  that  students  who  completed  English  111  with  the  TICCIT  system  scored 
about  two  points  lower  on  the  objective  posttest  than  similar  students  in  en- 
tirely lecture-discussion  classes.    But  the  magnitude  of  the  effect  varied 
according  to  students'  scores  on  the  objective-pretest,  and  there  should  again 
be  several  estimates  reflecting  the  range  of  effects.    The  estimated  effect 
begins  to  change  direction  for  students  whose  objective  pretest  score  exceeded 
30:    t  -  .01  when  the  objective  pretest  score  was  approximately  one  standard 

33l 


deviation  above  the  mean.    Only  for  students  with  nearly  the  maximum  possible 
score  on  the  objective  pretest  would  the  estimated  effect  be  positive  and 
significant.    The  significant  negative  effect,  however,  applies  to  over  half 
of  the  students  who  completed  English  111  with  some  T1CC1T  use  in  the  spring 
quarter.    The  estimated  effect  would  be  -4.41  points  for  students  with  objec- 

v. 

tive  pretest  scores  about  one  standard  deviation  below  the  mean  observed  in 
the  T1CC1T  condition.  * 

The  other  significant  treatment  effect  at  Alexandria  occurred  on  the  essay 
'pbsttest  in  the  fall  quarter.    That  regression  model  has  an  interaction  term 
indicating  different  effects  in  the  day  and  evening  divisions: 

TE  "  *420  for  students  completing  evening  T1CC1T  classes  and 

TD  *  **20  +  (-«56°)  "  -.14  for  students  completing  day  T1CC1T  classes. 
The  positive  effect  in  the  evening  division  represents  a  sizeable  point  dif- 
ference between  conditions  on  the  essay  posttest.    However,  some  of  this 
effect  might  be  attributed  to  instructors  rather  than  treatments.  One 
instructor  apparently  stressed  writing  skills  and  sacrificed  student  per- 
formance on  essay  tests  in  a  lecture-discussion  class  (see  Appendix  M,  the 
descriptive  statistics  for  section  14  in  the  fall  quarter).    Just  as  the 
negative  impact  on  essay  performance  at  Phoenix  could  be  explained  by  certain 
instructors'  lack  of  familiarity  with  the  T1CC1T  program,  the  positive  impact 
observed  at  Alexandria  may  be  a  function  of  an  instructor's  emphasis  in 
teaching  composition. 

The  one  consistent  result  from  the  analyses  for  student  achievement 
concerned  point  differences  between  conditions  on  the  objective  test  of  writing 
skills.    The  T1CC1T  program  led^to  higher  scores  on  the  objective  posttest, 
and  the  point  differences  were  in  the  neighborhood  of  two  points  or  about  five 
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percent  of  the  total  . score  possible.  _  Such  an  estimated  treatment  effect  may 
seem  negligible  in  terms  of  its  educational  importance  but  still  represents 
\a  significant  and  uncommon  finding.    The  effect  arose  from  the  application  of 
't"he  TICCIT  program  as  a  primary  instructional  resource.    Instructors  were 
responsible  for  directing  students  and  often  supplemented  the  TICCIT  curriculum 
by  holding 'small  group  discussions.    The  courses  in  which  the  TICCIT  program 
had  this  positive  impact  were  English  019  and  English  029  at  Phoenix.  These 
courses  placed  emphasis  on  the  fundamentals  of  written  English,  and  the  signi- 
'   fi cant  treatment  effects  indicated  TICCIT's  positive  impact  on  precisely  that 
component  of  student  performance,  namely,  the  objective  test  of  writing  skills. 

There  was  also  a  case  of  a  significant  treatment  effect  on  |est  performance 
in  the  spring  quarter  at  Alexandria.    The  TICCIT  program  as  an  adjunctive 
instructional  resource  had  a  negative  impact  on  student  achievement:  students 
completing  English  111  with  some  exposure  to  the  TICCIT  materials  had  lower 
scopes  on  the  objective  test  of  writing  skills  than  similar  students  from 
lecture-discussion  classes.    The  size  of  the  point  difference  varied  according 
to  entrance  ability,  defined  by  scores  on  the  objective  pretest.    This  same 
trait-treatment  interaction  between  entrance  ability  and  the  TICCIT  condition 
had  occurred  in  the  analyses  for  course  completion  as  a  program  outcome.  With 
that  dependent  variable  the  interaction  entered  into  the  regression  models  for 
j-  the  winter  and  fall  quarters  ,  (see  Table  A  .9)  •    Now,  with  student  performance  on 

the  objective  posttest  as  the  dependent  variable,  we  find  evidence  of  the  inter- 
action  in  the  spring  quarter  as  well.    Thus,  the  TICCIT  program  applied  as  an 
.  adjunct  to  conventional  practices,  the  condition  under  which  its  course  completion 
rates  came  closest -to  that  of  lecture-discussion  classes,  had  a  negative  impact 
on  some  aspect  of  student  performance  across  quarters  of  Alexandria,  and  that 
effect  worsened  as  scores  on  the  objective  pretest  fell. 
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5.3.c    Immediate  Learning  and  Retention 

Besides  the  posttests  which  served  as  measures  of  cumulative  learning, 
there  were  topical  tests  and  tests  for  retention.    These  tests  defined  student 
achievement  in  terms  of  immediate  learning  and  retention,  and  therefore  broadened 
our  evaluation  of  student  achievement  in  math  courses.    The  topical  tests  were 
taken  by  students  in  math  courses  as  soon  as  they  completed  instruction  in  a 
particular  topic.    The  topic  tested  in  introductory  algebra  was  linear  equations, 
and  in  intermediate  algebra  it  was  algebraic  expressions.    The  tests  for  reten- 
tion, were  actually  pretests  serving  a  dual  purpose.    Pretests  administered^: 
the  beginning  of  more  advanced  courses  in  -an  algebra  sequence  acted  as  both  a 
measure  of  entrance  ability  in  that  advanced  course  and  a  measure  of  retention 
for  the  prerequisite  course.    For  a  student  enrolled  for  Algebra  I  and  Algebra 
II  In  successive  academic  terms  the  pretest  for  Algebra  II  would  also  be  a 
retention  test  for  Algebra  I. 

The  results  of  the  analyses  for  topical  testing  appear  in  Table  5.14. 
Generally  the  estimated  treatment  effects,  represented  by  the  regression  weights' 
for  the  condition  variable,  follow  the  pattern  of  effects  observed  for  cumulative 
learning  (see  Tables  5.5  and  5.6).    There  were  two  cases  in  eleven  of  a  signifi- 
cant, positive  treatment  effect  on  posttest  scores  for  common  objectives;  there 
was  one  instance  in  six  of  a  significant,  positive  treatment  effect  on  immediate 
learning.    But  the  simplified  regression  models  in  Table  5.14  clearly  indicate 
that  a  sizeable  point  difference  separated  the  TICCIT  andSecture  conditions. 
The  TICCIT  program  had  a  positive  effect  of  over  two  points  on  students' 
immediate  learn Ag  in  four  cases,  and  such  an  effect  is  equivalent  to  ten 
percent  of  the  total  possible  score  on  topical  tests. 
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Achievement  Results:  Analysis  for  Immediate  Learning 
Alexandria  and  Phoenix 

Math  Courses 
Academic  Year  1975-76 


^4 


Matf  |)07 


Math  31 


Tern 

Spring 
Semester 


Math  108  Spring 
Semester 


Math  31  Fall 

X   .  Quarter 

•Math  31   ^  Winter 

'  Quarter 


Spring 
Quarter 


Math  32  Spring 
Quarter 


Dependent 
Variable 

Topical  Test 
Linear  Equations 
(Form  607) 

Topical  Test 
Algebraic  Expressions 
(Form  608) 

Topical  Test 
Linear  Equations 
(Form  617) 

Topical  Test 
Linear  Equations 
(Form  617) 

Topical  Test 

Linear  Equations  /' 

(Form  617)    — ' 

Topical  Test 
Algebraic  Expressions 
(Form  608) 


Number  of  Multiple 
Students  Correlation 


Independent  J  Regression  Value  of 
Variables        Weights  t-statistic 


72 


47 


63 


47 


27 


52 


,71 


.60 


.62 


.63 


.75 


.30 


TICCIT(1)/Lecture(ff)y  2.76 

Pretest  .23 

Constant  5.54 

TICCIT(1)/Lecture(0)  .  2.27 

Pretest  .25 

6.69 

TICCIT(1)/Lecture(0)  .41 

Pretest  .24 

Constant  5.73 

TICCIT(1)/Lecture(0)  2.17 

Pretest  .27 

Constant  3.38 

TICCIT(1)/Lecture(0)  2.61 

Pretest  .22 

Constant  ^6.81 

TICCIT(1)/Lecture(0)  -2.09 

Pretest  .39 

Constant  12.88 


3.16 
6.03" 


** 
*** 


U2, 
4.54' 


*** 


,3'5 
5.98' 


*** 


1.83 
5,( 


*** 


1.61 
4.75 


*** 


•1.61 
1.88 


** 


p<.01 
£  p<,001 


335  *** 


Table  5.14 
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9 

This  prevalent,  positive  impact  of  the  TICCIT  program  provides  an  important 
piece  of  evidence  about  student  achievement.    Posttesting  was  dependent  on  course 
completion,  and  any  analysis  based  on  the  posttests  would  be  susceptible  to  doubt 
given  tffe  discrepancies  in  coiirse  completion  by  condition.    But  topical  testing 
took  place  earlier  in  math  courses,  and  its  results  apply  to  a  wider  range  of 
students.    The  relevant  population  would  be  students  capable  of  making  progress* 
toward  course  completion  on  the  TICCIT  system.    Although  the  population  is  still 
not  the  same  as  all  students  enrolled  for  a  course,  it  is  a  less  restrictive  on 
than  that  capable  of  completing  a  math  course  under  the  TICCIT  program.     And  for* 
this  population  the  estimated  treatment  effects  were  predominately  positive  and 
of  appreciable  size. 

Table  5.15  presents  the  analyses  for  retention*    There  was  one  significant 
treatment  effect  and  it  was  negative.     Indeed,  the  two  slightly  positive  esti- 
mated effects  together  with  the  negative  effect  seem  to  contradict  other  results 
for  student  achievement.    But  often  students  began  a  more  advanced  course  on  the 
TICCIT  system  while  completing  its  prerequisite  course.     They  might  then  take  a 
course-specific  test  for  retention  prior  to  studying  the  mathematical  rules  and 
concepts  covered  in  the  test.    There  were  at  least  five  students,  for  example, 
who  started  Math  106  while  finishing  Math  007.     Such  cases  naturally  interfered 
with  our  testing  for  retention  and  may  invalidate  these  particular  results. 

5.3.d    Estimated  Treatment  Effects 

Table  5.16  gives  a  summary  of  the  treatment  effects  estimated  for 

* 

student  achievement.     All  of  these  estimates  refer  to  point  differences  in  post- 
test  performance  between  students  completing  a  course  under  the  TICCIT  program 
and  similar  students  in  lecture-discussion  classes.    There  were  significant 
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Achievement  Results:  Analysis  for  Retention 
Alexandria  and  Phoenix 
&^AeniaticB  Courses  / 


/^cijeflic  Year  1975-76 


Number  of  Students 
Multiple  Correlation 
>>  Standard  Error  of  Estimate 

Independent  Variables 

Condition: 
TICCIT (1) /Programmed (-1 ) /Lecture (0) 
Lecture(l)/Programmed(-1)/TICCIT(0) 

TICCIT(l) /Lecture  (0) 


Math  31 
to  Math  32 

N  »  44  (13  TICCIT) 

R  -  ,49 


se  „  »  2.54 
\  est 


Regression    Value  of 
Weight  t-statistic 


.06 
.45 


.11 
-  .82 


Math  007 
to  Math  106 

N  -  83  (17  TICCIT) 

R  ■  .50 


se  „  ■ 
est 


2.61 


Regression    Value  of 
Weight  t-statistic 


.55 


.78 


Math  106  or  108 
to  Math  117 

N  -  26  (15  TICCIT) 

R-  .71 

se  t  •  1.88 
est 

Regression    Value  of 
Weight  t-statlstic 


•Ml 


■2.17 


Entrance  Ability: 
Pretest 


.11 


3.16 


** 


.14 


5.07 


*** 


.18 


4,45 


*** 


Term: 

Fall  to  Winter (1) /Winter  to  Spring(O)  .05 

Fall  to  Spring(l) /Spring  to  Summer (0) 

Previous  Courses: 
Math  106(1) /Math  107(0) 


Constant: 


11.51 


.06 


-1.05 


9.02 


-1.44 


-2.34  -2.63 
7.82 


*p  <  .05 


** 


p  <  .01 


*** 


p  <  .001 


ERJCS 
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Estimated  Treatneot  Effects  on  Student  Achievement 


College 

Ten 

Course 

Effect  (t) 

t-Statistic 

Population 

Phoenix 

i 

Pall  4  Spring  1975-76 
Spring  &  Sunner  1976 

Spring  1976 
Pall  1976 

Math  007 

Math- 007 

Math. 106 

Math  108 

Math  117 
,  Math  117 

English  19,  29 

English  19,  29 

English  19 
,  English  29 

English  19,  29 

-  .07 

5,40 
3.37 

.97 
-1.74 
6.14  . , 

,20e 
2.38° 

.08e 

-  .50e 
1.68° 

■  A 

2,69** 

2.07* 
.43  • 
-1.13 

2.05* 

1.42  . 

2.48*  ' 

.68 
-3.03** 

2.41* 

Students  in  Day  TICCIT  Classes 
Students  in  Evenine  TlCCIT  ciiui* 
Students  in  TICCIT  Classes 
Students  in  TICCIT  Classes 
Students  in  Day  TICCIT  Classes 
Students  in  Evening  TICCIT  Classes 
Students  in  TICCIT  Classes 
Dependent  on  Age 
Students  in  TICCIT  Classes 
Students  in  TICCIT  Classes 
Students  in  TICCIT  Classes 

Alexandria 

Fall,  Winter  6  Spring  1975-76 

Winter  &  Spring  1976 
Winter  1976 

Spring  1976 

Fall  1976 

Math  31 
Math  31 
Math  32 
English  Ul 
English  111 
English  111 
English  111 
English  111 
English  111 
English  111 

 _„j 

4.40 

-  .11 

4.31 
.23e 

-  .93' 
.  -.16e 
-2.21* 
-.14e 

,42e 
-.90° 

3*24** 

2.15* 
1.00 

-  .68 
-1.17 
-1.96* 

-  .89 
2.11* 
-1.14 

Students  in  Day  TICCIT  Classes 
students  in  Evening  TICCIT  Classes 
Students  in  TICCIT  Classes 
Students  in  TICCIT  Classes 
Students  in  TICCIT  Classes 
Students  in  TICCIT  Classes 
Dependent  on  Ability 
Students  in  Day  TICCIT  Classes  • 
Students  in  Evening  TICCIT  Classes 
Students  in  TICCIT  Classes 

Effect  on  essay  posttest. 


Effect  on  objective,  posttest. 
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positive  effects  in  five  math  courses  of  six.    But  these  effects  became  evident 
only  when  analyses  encompassed  data  from  multiple  academic  terms.  Otherwise 
the  arbitrary  time  constraints  of  a  single  term  precluded  the  use  of  posttests 
taken  after  the  formal  close  of  the  term,  and  many  students  on  the  TICCIT  system 
chose  to  take  advantage  of  its  self-paced  feature  by  continuing  a  course  beyond 
a  term' 8  end.    There  were  also  significant,  positive  effects  on  the  objective 
pdsttest  of  writing  skills.    These  occurred  in  both  the  spring  and  fall  semesters 
at  Phoenix.    However,  study  on  the  TICCIT.  system  as  an  adjunct  to  traditional 
classroom  practices  led  to  a  significant  negative  effect  on  the  objective  posttest 
for  English  111  in  tire  spring  quarter. ^ 

To  some  extent  the  significant  effects  reflect  the  context  of  the  demonstra- 
tion as  well  as  outcomes  of  the  TICCIT  program.    The  sizeable  point  differences 
observed  in  Math  007  and  Math  117  occurred  in  the  evening  division.    That  was 
when  regular  faculty  members  supervised  students  on  the  TICCIT  system  while 
primarily  part-time  instructors  taught  the  lecture  sections  of  math  courses. 
The  significant  negative  effect  found  in  English  029  in  the  fall  semester  was 
also  related  to  staffing..  Part-time  instructors  were. responsible  for  both  UICCIT 
and  lecture-discussion  classes.    But  those  instructors  given  TICCIT  classes  were 
unfamiliar  with  the  program  and  assigned  too  late  to  receive  an  adequate  orienta- 
tion to  its  coverage  and  structure.    The  positive  impact  on  essay  test  performance, 
x    of  .42  for  English  111  in  the  fall  quarter,  might  also  be  attributed  to 
differences  among  instructors  rather  than  programs.    There  was  an  instructor  in 
the  evening  division  who  stressed  writing  skills,  and  tfiose  lecture-discussion 
class  had  low  essay  scores  relative  to  other  lecture-discussion  classes  and 
TICCIT  classes.    Since  students  from  this  class  constituted  nearly  half  of  the 
lecture-discussion  comparison  group,  their  essay  performance  was  enough  to  make 
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a  TICCIT  effect  apparent  in  the  evening.  Across,  day  and  evening  classes,  however, 
the  estimated  treatment  effect  would  not  be  Significant,  t  -  .06  and  t  -  .53. 

Even  if  we  were  to  disregard  all  significant  effects  somehow  linked  to 
particular  facts  about  the  context  of  the  demonstration,  there  would  still  be  * 
six  other  cases.    The  positive  impact  of  the  TICCIT  program  on  student  achieve- 
ment in  math  appears  in  Math  106,  Math  31  and  Math  32.    The  positive  impact  of 
the  TICCIT  program  on  student  achievement  in  English  appears  in  scores  on  the 
objective  test  of  writing  skills:    significant  effects  were  evident  in  English 
019  and  English  029  for  both  the  spring  and  fall  semesters.    However,  there 
was  a  significant  negative  effect  in  English  111  that  became  more  pronounced 
for  students  of  lower  entrance  ability.    This  estimated  treatment  effect  comes 
from  the  spring  quarter,  the  one  quarter  for  which  there  had  not  been  a  signi- 
ficant negative  effect  on  course  completion  for  English  111  and  the  completion 
effects  that  had  occurred  in  the  winter  and  fall  quarters  also  varied  according 
to. entrance  ability  (see  Table  4.9). 

This  raises  a  critical  question  for  which  there  is  really  no  adequate 
answer:    Does  the  negative  TICCIT  impact  on  course  completion  make  favorable 
outcomes  more  likely  for  other  aspects  of  student  performance?    Since  the  TICCIT 
program  acted  to  select  only  certain  students  for  posttesting,  it  may  be  that 
characteristics  other  than  those  included  in  the  achievement  analyses  distin- 
guished students  in  the  TICCIT  condition  from  those  in  the  lecture  condition. 
Then  such  characteristics  might  explain  differences  in  posttest  performance 
seemingly  attributable  to  the  treatment.    For  example,  the  negative  impact  on 
course  completion  found  across  academic  terms  (see  Table  4.8)  might  account  for 
differences  in  student  performance  on  the  objective  posttest  in  English  019  and 
English  029. 
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The  pronounced  differences  by  condition  in  math  completion  rates  (see  Tables 
4.2  and  4.3)  make  the  selective  nature  of  the  TICCIT  program  an  attractive  inter- 
pretation for  the  achievement  results.    But  other  data  tend  to  support  acceptance 
of  the  positive  treatment  effects  as  program  outcomes.    Testing  achievement  in 
particular  topics  revealed  a  positive  TICCIT  impact  at  stages  in  math  courses 
prior  to  posttesting  (see  Table  5.14).    Yet  data  f roip  postteating  all  students 
in  Algebra  I  and  Algebra  II,  regardless  of  their  anticipated  course  grades,  indi- 
cated a  negative  TICCIT  effect*    Comparisons  of  students . attending  math  classes 
toward  the  close  of  a  quarter  resulted  in  these  estimates:     x  -  -4.93  and  t  ■  - 
4.34,  £  <  .001  for  Math  31  in  the  fall  quarter;  x  -  2.30  and  t  -  1.18  f or  M*th 
31  in  the  winter  quarter;  x  -  -3.80  and  t  -  -1.59  for  Math  31gp^  the  spring 
quarter;  and  x  -^-1.35  and  t  -  -.99  for  Math  32  in  the  winter  and  spring  quarters 
These  comparisons  do  incorporate  a  bias,  however,  in  the  nature  of  the  groups 
posttested  in  each  condition.    Students  still  attending  lecture  classes  were 
those  completing  a  math  course  while  students  from  TICCIT  classes  were  primarily 
those  making  slow  progress  in  a  math  course.    With  such  comparison  groups  a 
negative  estimate  of  TICCIT's  effects  should  be  expected. 

It  is  perhaps  easier  and  more  appropriate  to  accept  the  estimates  given 
in  Table  5.16  for  what^rhey  represent.  Each  estimate  of  a  treatment  effect  ■ 
reflects  a  point  difference  in  posttest  performance  between  students  who 

completed  a  course  under  the  TICCIT  program  and  similar  students  from  lecture- 

v 

discussioq  classes.    Similarity  was  determined  by  entrance  ability  as  well  as 
demographic  characteristics.    Our  definition  of  similarity  perhaps  omitted  a 
critical  variable,  but  we  remain  uncertain  about  what  if^j^ht  be  and  how  it 
might  be  measured. 


344 


5.4    Discussion  >  .  * 

The  TICCIT  program  had  a  positive  impact  on  student  achievement.  Students" 
capable  o?  completing  a  course  under  the  TICCIT  program ^generally  attained  higher 
posttest  scores  than  those  reached  by  similar  students  under  conventional  teaching 
methoda.    These  results  were  evident  in  those  applications  of  the  TICCIX.  program 
closest  to  the  developers'  concept  of  mainline  it>stp»ctioni    ?iit  such  "applications 
had  Also;  led  to  pronounce^  differences  ins  course^  completion  r»tes  unfavorable  to 
t^e  TICCIT  program.    ■    ,        .  ■  0 

ItVtaigtft  be  appropriate  to  insider  whethe?  the  favorable  AchiWient  dif- 
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.  ferenc^  Pf-f  s^et  the  negat^P  Tirn-T  ^m^«,^  ^«  ^x.  1 „^  _A    rm      _^  . 


negative' TICCIT  impact  on  bourse  completion^    The  significant 
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point  differences  in'  math  courses  of  tin  exce|tied  ten  percent  of  the  'to.tal'score 

••  *  *  jtC  '  •'  *Jf       ■  '  '  •        ^  ■  ' « *  • 

^  p>ssibp"b4  a„  posttest  and  Corresponded  td  approximately,  two- thirds  of  a  standard 
deV!^"0tl^^  teat~    This  wajs  the  case  ift^jie  beginning  algebra  couJrsetT  as 
we^l  as  ^h^^r>  advanced  algljlfra ^  o^ses.^ff  ered  qnTthl^llciT  system.    But.  even 
greater  a^Sement •  dif ff r^n^sj^iid'd^ot  balance  the^discrepancies  between-, 
condition*  inithe^^tyietion  rates  jkx^  m&th  courses.    A.  five  point  advaniag. 


V «   in  posttest  i3Cfere8,hardlt  seems  ^rth^K4rtp  iji  completion  rates  t\  half  pf\he 
*are  attainab  le  through  ^/convent  ional  Rrapt  ice^ .    P^hap^he-pdsit  i^e  achievement 


effect 8  found  lij  mat 
onl^  W omi8e|^r^aF 


3e8^ep^se^f^he  promise  <mtHe  TICCIt  program,  but 
pons  witlv.lSartfers  capable  of  vj^dtttotendent  study  or 


under  c^r^imi5tatfc§B  wherl  Vn  outcome  like  completion  vould  not  be  a  factor. 

The  achievement "results  obtained  in  English  courses^  suggest  a  different  con- 
clusion.   Adjunctive  use  of  the  TICCIT  program,  even  carefully  integrated  with 
^classroom  prattices,  letl^to  significant  ^ativej  effects  on^eit;hei:> course  com- 
Vl^ion  or  student  achievement.    The  estimated  Jlze  of  these  effects  varied 
aqp.drding  t^, students'  entrance  ability.    St^<|B|?8  With  lower  pretest  scores 
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stood  to  lose  more  as  a  result  of  exposure  to  the  TICCIT  program  instead  of 
traditional  instruction.    Conversely,  a  select  group  of  students  with  high 
pretest  scorea  benefitted  from  working  on  the  TICCIT  system.    But  use  of  the 
TICCIT  program  as  a  supplement  to  classroom  teaching  failed  to  demonstrate 
the  same  consistency  of  positive  achievement  effects  found  in  math  courses. 

It  was  with  a  mainline  application  of  the  TICCIT  program  to  instruction 
in  English  composition  that  positive  achievement  effects  began  to  balance 
negative  completion  effects.     There  were  consistent  and  statistically  signi- 
ficant  point  differences  on  the  objective  test  of  writing  skills.  They 
occurred  when  students  studied  the  fundamentals  of  English  on  the  TICCIT 
system.    Often  instruction  was  supplemented  by  small  group  discussions  about 
common  difficulties  in  learning  English  fundamentals  as  well  as  writing 
assignments.     Instructors  took  an  active  role  in  prescribing  material  for 
study,  leading  group  discussions,  and  handling  individual  student  problems. 
Even  student  performance  on  essay  tests  was  better  in  the  TICCIT  condition, 
except  when  part-time  instructors  unfamiliar  with  the  TICCIT  program  conducted 
TICCIT  classes.    These  achievement  results  not  only  show  promise  for  the 
TICCIT  program  but  raise  hopes  that  an  improved  program  would  yield  completion 
^ates  at  least  comparable  to  lecture-discussion  instruction  and  achievement 
outcomes  still  better  than  those  from  this  evaluation. 

Finding  positive  treatment  effect^pon  test  performance  in  English  surprised 
us.     Prior  studies  of  computer-assisted  instruction,  especially  at  the  college 
level,  suggest  that  this  teaching  method  might  benefit  highly  structured  subject 
fields.    But  perhaps  that  structure  also, acts  as  a  deterent  to  student  progress 
in  a  self-paced  system.     A  subject  less  well  defined  might  be  conducive  to 
greater  teacher  participation  and  more  realistic  expectations  of  computer 

346 


>  299  - 


capabilities  for  teaching,    without  *  definite  structure  to  Che  subject  matter 
there  is  also  opportunity  £o*  student  choice  and  teacher  direction.    And  the 
rigor  of  developing  computer  currieulet  materials  would  probably  be  beneficial 
in  forcing  specificity  and  structure  on  teaching  ill-defined  subject*.  Perhaps 
we  should  puraue  techniques  like  computer-assisted  instruction  for  teaching  In 
the  humanities,  social  sciences  and  other  similar  fields  with  aa  much  energy 
and  excitement  as  usually  devoted  to  disciplines  known  for  their  structure. 

Besides  the  arguments  already  set  forth  as  alternative  interpretations  of 
our  est ima ted *  treatment  effects,  there  is  another  plausible  explanation  that 
deserves  mention.    Tobias  (Tobias,  19?6;  Tobias  &  Ingber,  1976)  reviewed  attempts 
to  adapt  instruction  to  individual  differences.    He  concluded  that  students  with 
less  prior  familiarity  of  the  subject  matter,  or  lower  entrance  ability,  need 
more  instructional  support.    This  view  0f  achievement-treatment  interactions 
fits  well  with  the  results  of  our  evaluation.    It  would  be  consistent  with  the 
selective  nature  of  the  TICCIT  program  In  math  courses:    students  weak  in 
entrance  ability  failed  to  satisfy  course  .requirements  within  an  academic 
term  while  students  strong  in  prior  familiarity  completed  th*  course  and  did 
well  on  posttests.    Interactions  between  pretest  score  and  instructional 
condition  did  come  into  our  regression  yodels  for  completion  or  achievement 
in  English  courses.    The  TICCIT  program  may  provide  instructional  support 
sufficient  for  students  of  strong  entrance  ability  but  inadequate  for  others. 
^Accepting  achievement- treatment  interactions  as  a  viable  interpretation  of 
results  would  also  imply  that  our  analyses  included  the  key  student  charac- 
teristic in  identifying  similar  students,  namely  entrance  ability. 
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Subject  to  ^tXi^em^nt- treatment  interactions,  the  TICCIT  program  succeeded 
in  Improving  stut|*At  achievement  %    Students  who  completed  courses  under  the 
TlCClT  program  g^ti*f*Uy  attained  higher  posttest  scores  than  comparable  students 

did  in  lecture-d^^aion  classes*    This  effect  was  observed  among  those  students 

•  \  ■  J 

better  prepared  £0*  their  mathematics  or  English  courses  in  terms  of  their  prior 

§  ' 

familiarity  of  %e  subject  matter •    While  such  students  may  constitute  a  small 
percentage  of  tty  ^o*munity  college's  total  enrollment,  especially  for  courses 
in  mathematics,  #b*y  might  be  a  majority  of  the  learners  in  another  context* 
At  industrial  or  educational  institutions  which  screen  and  select  applicants  on 
a  competitive  ba^i^i  the  TICCIT  program  could  result  in  uniform  positive  effects 
on  achievement**  * 


Chapter  6 

■a 

Student  Attitudes 

Over  the  last  decade  there  has  been  a  dramatic  increase  in  the  role  of 
student  attitudes  in  evaluating  courses.    Colleges  now  actively  solicit 
student  Jireactlons  to  their  courses  and  Instructors  through  the  routine  ad- 
ministration  of  questionnaires.    Students'  reactions  to  their  courses  have 
become  an  important  factor  in  considering  curriculum  changes,  and  their 
ratings  of  instructors  Influence  decisions  about  faculty  appointment.  * 
Despite  the  rise  in  the  recognition  of  student  attitudes  as  a  legitimate 
concern  in  instruction,  curriculum  evaluations  often  neglect  this  aspect 
of  student  performance.    Yet  we  can  assess  attitudes  across  instructional 
conditions  and  make"  comparisons  of  program  results. 

Such  comparisons  seem  especially  appropriate  in  evaluating  computer- 
assisted  instruction.    Much  of  the  interest  in  this  technique  evolved  from 
its  promise  for  tailoring  instruction  to  each  individual  student.  Cosputer- 
assisted  instruction  would  presumedly  enable  students  to  learn  at  a  pace 
consistent  with  their  ability  and  readily  adapt  to  the  needs  of  students. 
So  students  should  prefer  learning  on^computers  to  the  teaching  in  class- 
rooms.   Research  results  indeed  support  this  interpretation:    There  is  an 
abundance  of  studies  showing  positive  student  attitudes  toward  computer- 
assisted  instruction  (e.g.,  Brown  &  Gilman,  1968;  Hathis  et  al.  1970;  Smith, 
1973;  Summerlin,  1971).    But  these  deal  with  other  computer  systems  and 
concern  students  at  other  levels  of  education.    We  sought  to  document 
student  reactions  to  the  TICCIT  program  at  community  colleges,  and  to 
evaluate  these  affective  results  in  relation  to  the  reactions  of  similar 
students  in  lecture-discussion  sections  of  the  same  courses. 
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This  chapter  addresses  student  attitudes  as  a  third  dimension  of 
studsnt  performance.    The  first  section  specifies  the  student  population  for 
the  analysis  of  attitude  results;  it  also  presents  an  explanation  of  the 
measures  that  define  student  attitudes  for  our  evaluation.    The  second 
section  presents  summaries  of  the  attitude  data.    These  summaries  reflect 
both  students'  scores  on  the  overall  attitude  scale  and  their  responses  to 
specific  statements  on  the  attitude  survey.    Analyses  of  the  attitude  data 
and  estimates  of  treatment  effects  appear  in  the  third  section  of  this 
chapter.    Like  the  analyses  for  student  achievement  which  concentrated  on 
posttest  items  reflecting  objectives  held  in  common  across  instructional 
conditions ,  the  analyses  here  focus  on  those  survey  statements  that  were 
identical  or  closely  parallel  in  TICCIT  classes  and  lecture-discussion 
sections.    The  final  section  is  a  discussion  of  the  attitude  results  and 
their  possible  implications. 
6.1    Students  and  Attitudes 

As  with  course  completion  rates  and  student  achievement ,  evaluating 
the  impact  of  the  TICCIT  program  on  student  attitudes  depended  on  drawing 
contrasts  across  comparison  groups.    Students'  attitudes  toward  courses 
taught  in  a  conventional  manner  served  as  relative  standard  against  which  we 
could  estimate  TICCIT  effects.    In  both  TICCIT  classes  and  lecture  sections 
students  responded  to  a  survey  about  their  course.    They  gave  their  Judg- 
ments of  the  course  and  their  reactioq^to  its  instruction.  However, 
instruction  provided  by  teachers  and  instruction  conducted  by  computers  may 
be  quite  different.    The  attitude  surveys t  like  the  achievement  posttests 
that  reflected  objectives  held  in  common  across  conditions,  focused  on 
those  aspects  of  a  course  which  appeared  in  lecture  sections  as  well  as  in 
TICCIT  classes. 
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The  titlittad  triitnot  effects  prtitntid  here  mey  seem  unfamiliar. 
Although  •valuations  frequently  define  a  program's  Impact  on  achlavamsnt  as 
a  number  of  points  on  a  test,  the  same  approach  Is  seldom  applied  to  the 
problem  of  assessing  attitudes.    But  attitudes  represent  another  cooponent 
of  student  performance  which  may  be  affected  by  a  program*  Computer- 
assisted  instruction  especially  may  appeal  to  students  for  its  innovative 
delivery  of  instruction  tailored  to  the  individual  learner.    He  applied  the 
same  procedures  to  evaluating  course  completion  rates,  student  achievement, 
and  student  attitudes.    This  meant  that  the  same  degree  of  care  had  to  be 
devoted  to  defining  student  populations  and  to  developing  fair  bases  for 
comparisons. 

6.1.a    Population  of  Students 

Students  responded  to  our  attitude  survey  upon  completing  a  target 
course  in  aathematics  or  English.    This  practice  in  itself  restricted  the 
population  of  respondents  in  each  comparison  group.    It  has  already  >een 
*hown  that  there  were  differences  in  program  flNftcomes  according  to. the 
criterion  of  course  completion.    Generally,  one-half  to  two-thirds  of 
the  students  enrolled  in  lecture  sections  of  target  courses  completed  all 

course  requirements  within  an  academic  term.    A  lower  proportion  of  the 

G 

students  enrolled  for  the  same  courses  under  the  TICCIT  program  actually 
finished  their  studies  within  an  academic  term  and  received  college  credits 

or  the  course.    These  lower  completion  rates  were1  particularly  evident  in 
math  courses  and  also  apparent  in  English  courses.    Yet  those  students  who 
did  successfully  complete  a  course  under  the  TICCIT  program  attained  higher 
posttest  scores  than  comparable  students  in  lecture  sections.    How  do  these 
results  affect  inferences  about  student  attitudes?    An  answer  lies  in  the 
student  population  for  which  we  may  draw  Inferences.  . 
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Thm  procedures  for  eetlaatlnf  treatment  effects  take  diecrepaacies  la 

course  completion  rates  into  account  •    Consider  what  a  traatMnt  affact 
representa:    it  stands  for  tha  dlffaranca  between  tha  attltudaa  expressed  by 
a  atudant  completing  a  c ours a  undar  tha  TICCIT  program  and  tha  attltudaa 
that  tha  same  atudant  would  probably  hold  lb  a  lactura  •action.    Glvan  tha 
responses  of  atudant a  from  TICCIT  claaaaa,  tha  raaponaaa  of  similar  student^ 
in  lactura  sections^  become  tha  baat  availabla  estimates  of  how  TICCIT 
atudants  would  raact  in  the  lactura  condition.    Statistical  procaduraa  atart 
with  the  population  of  students  who  completed  courses  undar  tha  TICCIT 
program  and  responded  to  our  attitude  survey,  and  the  procedures  yield 
estimated  treatment  effects  that  apply  only  to  this  population. 

To  a  great  extent  the  strength  of  our  analyses  rests  with  the  def ini- 
tlcm  of  student  similarity  across  conditions.  When  student  achievement  is 
the  criterion,  achievement  pretests  and  demographic  characteristics  provide 

a 

an  adequate  definition  of  student  similarity.    Tfce  multiple  correlations  foe 
the  achievement  analyses,  usually  in  the  neighborhood  of  .60  and  sometimes 
above  .70,  attest  to  the  usefulness  of  these  variables  in  identifying  the 
lecture  counterparts  to  students  who  had  completed  courses  under  the  TICCIT 
program.    However,  there  was  no  Instrument  that  would  fulfill  the  role  of  an 
achievement  pretest  when  examining- student  attitudes*    Pilot  trials  with 
brief  surveys  of  students'  predispositions  toward  computer-assisted  instruc- 
tion had  convinced  us  that  such  attitude  pretests  were  too  susceptible  to 
suggestions  made  in  orientation  sessions  and  too  unreliable.    Students  h%d 
little  prior  exposure  to  computers  from  which  they  might  form  initial  opin- 
ions.   In  finding  students  from  lecture  sections  comparable  to  those  who 
had  completed  TICCIT  -classes,  we  relied  on  the  same  set  of  variables  in  the 
attitude  analyses  as  in  the  achievement  analyses. 
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This  consistency  in  the  approach  taken  to  data  analysis  should  not  bs 
regarded  as  an  implication  of  the  dependence  of  attltil^e  reeulte  on^achieve- 
Mnt  results.    That  stadent  achievemel^t  improved  under  the  TICCIT  program 
is  a  separate  finding ,  and  any  statement  of  atfeitudlnal  Impact  stands 
apart  from  it.    However,  our  estimates  of  treatment  effects  for  these  two 
components  of  student  psrformance  do  refer  to  the  eame  population:  those 
students  capable  of  completing  a  course  under  the  TICCIT  program.  Only 
for  this  group  of  students  should  the  treatment  effects,  both  those  on 
student  achievement  and  those  on  student  attitudes ,  be  regarded  as  repre- 
sentative. 

6.1.b    Measures  of  Attitudes  « 

Statements  on  the  student  survey  and  scales  formed  from  responses 
to  particular  statements  really  define  what  student  attitudes  meant  within 
the  context  of  this  evaluation.    Responses  to  an^ttitude  survey,  unlike 
answers  to  an  achievement  posttest,  possess  no  absolute  right  or  wrong  key. 
They  do  indicate  direction  as  each  response  to  a  statement  may  be  taken  as 
falling  on  a  continuum  of  negative  to  positive  affect.    Whether  stronger 
agreement  with  a  statement  indicates  a  more  or  less  positive  attitude 
toward  the  TICCIT  program  would  depend  on  the  phrasing  of  the  particular 
item  stem.    Moreover,  the  estimated  treatment  effects  represent  differences 
of  degree:    How  positive  or  negative  were  students'  reactions  to  a  course 
under  the  TICCIT  program  in  relation  tp;? student  attitudes  toward  that  course 
in  a  lecture-discussion  format? 

Student  Survey.    The  studenf  attitude  survey  was  meant  to  solicit 
reactions  and  opinions  about  the  instruction  provided  in  a  course  (see 
~ Appendix  f).    Emphasis  naturally  fell  on  aspects  of  the  TICCIT  program  and 
their  parallels  in  traditional  classes.    There  were  statements  about  the 
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features  of. learner  control  built  Into  the  TICCIT  system  (e.g.,  items  7-11 
and  13  on  Form  035) ,  about  student  participation  in  the  process  of  learning 
(e.g.*  items  1,  5,  lVand  16  on  Form  035),  and  about  instruction  tailored  to 
the  individual  student  (e.g.,  items  2,  3,  6  and  18  on  Form  035).    Some  of 
these  questions  had  been  adapted  from  other  instruments  dealing  with  students' 
attitudes  toward  computer-assisted  instruction  (Brown,  1966)  or  with  students' 
ratings  of  college  teachers  (Centra,  1972).    Most  questions  were  developed 
especially  for  this  evaluation  so  as  to  cover  specific  features  and  possible 
advantages  or  disadvantages  of  the  TICCIT  program. 

Response  Format.    Each  item  stem  intended  to  solicit  student  attitudes 
has  a  five-point  response  scale.    Students  could  indicate  their  extent  of 
agreement,  from  strong  agreement  coded  as  five  points  to  strong  disagreement 
coded  as  one  point,  with  an  item.  '  Thus  the  maximum  score  possible  for  20 
items,  for  example,  would  be  100  points,  and  this  would  reflect  a  highly 
enthusiastic,  positive  attitude.    Because  some  item  stems  had  been  phrased 
iti  a  negative  manner  (e.g.,  "The  method  of  institution  for^  this  ^d^^e  was 
too  impersonal  for  me")  in  order  to  discourage  the  simple  repetitio^of  the 
same  response  across  all  items ,  responses  to  these -statements  had  to  be 
reversed  in  assigning  scores.    Only  those  statements  on  the  attitude  survey 
with  a  clear  direction,  whether  jjjjj^B iti ve  or  negative,  entered  into  attitude 
scores.  f  ' 

It  was  possible  that  students  '„woul/i  react  favorably  to  certain  compo- 

-  •  •  J 

nents  of  a  course  under  the  TICCIT  program  and  unfavorably  to  others.  A 
simple  total  for  an  attitude  sqor^wbuld  fail  to  reveal  such  distinctions. 
By  conducting  factor  analyses  of  the  attitude  data,  we  identified  the  major 
dimensions  underlying  students'  reactions  to  courses  under  the  TICCIT  pro- 
gram.    Since  these  dimensions  would  become  the  basis  for   comparisons  of 
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results,  we  studied  responses  to  the  statements  that  appeared  both* on  the 
survey  given  in  TICCIT  classes  and  on  the  survey  given  in  lecture  sections. 

Factor  Analyses.    Each  factor  analysis  dealtf  with  an  item  intercorrela- 
£ion  matrix.    An  item  was  included  in  that  matrix  if  its  stem  had  a  definite 
direction  and  if  it  appeared  on  the  surveys  for  both  the  JJICCIT  and  lecture 
conditions.    The  student  survey  administered  in  mathematicl^ourses  under 
the  TICCIT  program  in  the  fall  term  of  the  1975-76  academic  year  (Form  035 
in  Appendix  F)  had  a  total  of  35  questions.    Twenty-njjne  of  these  questions 
(items  1-29)  were  actually  statements  accompanied  by  a  $4*ve-point  response 
scale;  the  other  questions  really  pertained  to  student  activities.    Seven  of 
the  statements  (items  2,  5,  7,  15,  21,  23  and  25)  had  no  direct  counterpart 
in  the  lecture  condition,  and  four  statements  (items  26-29)  had  no  obvious  • 
direction  but  instead  solicited  information  from  the  student's  perspective. 
This  left  18  items  to  be  examined  through  factor  analysis.    A  revision  of 
the  student  survey  for  mathematics  jjourses  led  to  the  addition  of  another 
item  in  the  form  for  the  second-half  of  the  1975-76  academic  year  (item  15 
in  Form  069).    There  were,  therefore,  19  items  in  the  intercorrelation 

for  that  factor  analysis.    The  student  survey  employed  in  Jtoglis h 
»es  included  the  same  statements,  referring  to  English  rati 
mathematics  courses,  as  well  as  two  items  adapted  from^an  instructor's  own 
questionnaire  (items  16  and  17  on  Form  068).    All  of  the  item  stfejfc  con- 
sidered  in  the  factor  analyses  appear  in  Table  6.1.  *      f  0 

The  three  factor  analyses  presented  in  Table  6. 1  correspond  to  the 
three  forms  of  the  student  survey  administered  in  the  demonstration  period. 
The  first  analysis  was  conducted  for  the  .survey  diatf|»tfted  to  students  in 
mathemati^l^courses  under,  the  TICCIT  program  in,  t]ffij££ll  term  of  the  1975-76 


academic  )Mri(7orm  035).     For  the  remainder  b^Elj^Jlicade^ic  year  students  / 


Student  Attitudes  in  Courses  under  the  TICCIT  Progran 
Variaax  Prinary  Factors 


hsthenatlcs  Courses 

Mathematics  Courses 

English  Courses 

Fsll  1975-76 

Spring  1975t76 

Spring,  Fall  1976 

(N  ■  126) 

(N  ■ 

159) 

(N  -  140) 

f 

Factors 

Factors 

Factors 

Fort  035 

Fori  069 

Fori  068  Stea 

A-I 

A-II  A-III 

R-T 

It-TT 

0  11 

C-I 

C-II  C-III 

1 

1 

1     In  thli  couru  I  felt  free  to  ask  questions . 

74 

DJ 

55 

3 

3 

3     I  received  a  lot  of  Individual  attention  fron  Instructors 

75 

A7 

80 

4 

4 

4     The  lnstructor(s)  seened  genuinely  concerned  with  ny  progress 

77 

]) 

-66 

6 

6 

6     Instruction  in  this  class  net  ny  own  particular  needs 

71* 

UJ 

52 

8 

8 

8     ADVICE  helped  k  to  progress  through  this  course 

63 

Jl 

62 

9 

9 

9     PRACTICE  helped  le  in  learning  rules  and  concepts 

80 

66 

10 

10 

10     HELP  provided  clear  explanations 

70 

71 

'75 

11 

11 

11     RULE  isde  concepts  easy  to  learn 

77 

62 

70 

12 

12 

12     Taking  tests  let  me  .know  whether  I  really  understood  the 

uterial 

52 

13 

13 

13     EXAMPLE  did  not  show  Be  hov  to  (boIvc  problens/vrlte  veil) 

-50 

14 

14 

14     In  this  course  I  felt  responsible  for  ny  own  learning 

16 

17 

19     In  this  course  I  felt  challenged  to  do  ay  beet  work 

59 

69 

59 

50 

17 

18 

20     I  was  satisfied  with  iy  personal  progress  in  this  course 

69 

64 

IB 

19 

21     The  Mthod  of  Instruction  for  this  course  was  too ' 

impersonal 

-59 

♦ 

-69" 

-59 

19 

20 

22     Hy  interest  in  (nsth/vriting)  has  been  increased  by  this 

< 

course 

67 

p 

54 

50 

20 

21 

23     I 'would  recomend  this  course  to  ny  friends 

76 

62 

57 

56 

22 

23 

25     Tiae  passed  quickly  while  I  was  in  class 

24 

25 

27    c  I  tried  to  just  flnlBh  the  lessons  rather  than  learn  the 

Mterlal 

-72 

15 

18     Other  students  in  this  section  seemed  to  like  the  course 

16     I  learned  a  lot  about  gramar  In  this  course 

58 

17     I  learned  a  lot  about  writing  in  this  course 

57 

* 

Only  entries  above  .49  appear  in  this  table.  Decinal  points  have  been  omitted. 
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in  TICCIT  classes  for  math  courses  responded  to  a  revised  form  of  the  survey 
(Form  069),  and  these  data  formed  the  item  intercorrelation  matrix  for  the 

m 

second  factor  analysis.    A  third  analysis  was  conducted  for  the  survey  (Form 
068)  given  in  English  courses  on  the  TICCIT  system;  but  only  for  courses  con- 
sistent with  the  concept  of  mainline  instruction. 

The  item  intercorrelation  matrix  for  each  of  these  suAey  forms  was 
subject  to  a  principal  components  analysis  with  varimax  orthogonal  rotations. 
In  each  analysis  a  break  in  the  magnitude  of  the  latent  roots  determined  the 
number  of  factors  to  be  rotated.    Three  factors  wsre  retained  for  rotation 
in  the  analysis  for  the  student  survey  distributed  in  mathematics*  courses  in 
the  fall  term,  labeled  as  Factors  A-I,  A-II  and  A-III  in  Table  6.1.  The 
eigenvalues  associated  with  these  factors  were  6.40,  1.66  and  1.31,  respec- 
tively.   The  two  factors  shown  in  the  varimax  solution  for  the  student  survey 
used  in  mathematics  courses  in  the  second-half  of  the  1975-76  academic  year, 
Factors  B-I  and  B-II,  had  eigenvalues  of  7.15  and  1.46.    The  analysis  of  the 
survey  given  in  English  courses  revealed  three  factors,  Factors  C-If  C-II 
and  C-III,  with  eigenvalues  of  6.01,  1.79  and  1.56,  respectively.  These 
varimax  solutions,  then,  accounted  for  nearly  50Z  of  the  total  variance  in 
each  intercorrelation  matrix.    The  solutions  themselves  appear  in  Table  6.1 
and  suggest  three  dimensions  of  students'  attitudes  toward  courses  under  the 
TICCIT  program. 

Dimensions .    Across  academic  terms  and  courses  there  is  a  remarkable 
consistency  to  the  factor  structure  shown  in  Table  6.1.    The  items  which 
loaded  highest  under  Factor  A-I  can  also  be  found  grouped  together  under 
Factors  B-I  and  C-I.    There  is  nearly  complete  overlap  among  Factors  A-III, 
B-II  and  C-III.    And  the  statements  under  Factor  A-III  also  appear  under 
Factors  B-I  and  C-II.    These  three  dimensions  reflect  important  components 
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*of  I  students',  attitudes  toward  'a  fioucije:     the  studies  personal  satisfaction 
with  the  course  fs  a  whole;  the  studenf/s  percept io^  of  the  individual  - 
att^^ion  given' to  him  or  her;  and  the  student's  reactions  to  specific 
features  of  the  TICCIT  system. 

The  statements  under  Factor  A-I  relate  to  a  student's  personal  satis- 
faction with  a  course  on  the  TICCIT  system.    These  items  concern  whether  the 
student  would  recommend  the  course  to  friends,  the  extent  to  which  instruc- 
tion met  the  student's  own  particular  n^eds,  the  degree  of  satisfaction  a 
student  felt  with  his  or  her  personal  progress  in  the  cbur^,  and  the  extent 
to  which  the  course  increased  the  student's  interest  in  the  subject  matter. 
Other  statements  shown  under  this  factor  refer  to  the  challenge  posed  by  a 
course  and  the  impersonal  nature  of  instruction.    All  of  these  item  stems 
appear  together  in  analyses  corresponding  to  mathematics  courses  in  the 
fall  term  of  1975-76  (Factor  A-I),  mathematics  courses  in  the  remainder  of 
that  academic  year  (Factor  B-I),  and  English  courses  In  the  1976  calendar 
year  (Factor  C-I).    Additional  statements  on  students'  own  estimates  of  the 
amount  they  had  learned  in  a  course  (items  16  and  17  on  Form  068)  also 
loaded  on  Factor  C-I.    This  factor  consistently  had  the  highest  eigenvalue: 
it  explained  the  largest  proportion  of  total  variance.    Because  it  reflects 
a  student's  evaluation  of  a  course  in  terms  of  his  or  her  own  progress  and 
needs,  this  dimension  seems  to  be  critical  for  any  assessment  of  student 
attitudes.    The  Items  under  this  factor  formed  the  scale  called  Course 
Satisfaction  in  <?ur  analyses  of  TICCIT  effects  on  student  attitudes. 

Factor  A-II  includes  those  statements  relevant  to  the  Individual  atten- 
tlon  that  students  received  in  a  course.    The  two  items  with  the  highest 
entries  under  this  factor,  whether  interpreted  as  shown  under  Factor  A-II, 
B-II  or  C-III,  pertain  to  the  individual  attention /fiven  by  instructors 
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(item  3)  and  their  concern  with  a  student's  course  progress  (Item  4).  Often 
such  Individual  attention  came  Into  play  when  students  raised  questions 
about  the  subject  matter  presented  on  the  TICCIT  system  or  asjced  about 
procedures  for  operating  the  TICCIT  system  (see  Chapter  7;    Student  Activi- 
ties).    Instructors  or  proctors  would  respond  to  these  student  requests  for 
assistance.    Thus,  it  is  consistent  with  the  affective  dimension  represented 
by  Factor  A-II  that  a  statement  about  students'  comfort  in  asking  questions 
(item  I),  questions  which  in  turn  elicited  individual  attention,  also  has  a 
high  entry.    Even  a  statement  about  the  individual  attention  which  students 
received  from  the  TICCIT  system  (item  8),  TICCIT's  conments,  attention  is 
based  on  interpersonal  contact.    This  factor  became  a  second  scale,  Individ- 
ual Attention,  for  our  analyses  of  affective  impact. 

The  third  dimension  of  studept  attitudes  covers  students'  reactions  to 
specific  features  of  the  TICCIT  system.    Factor  A-III  consists  of  items 
related  to  the  learned-control  options  called  PRACTICE,  HELP  and  RULE  (items 
9,  10  and  11).    In  the  factor  analysis  conducted  for  attitudinal  data  from 
English  courses,  axparallel  factor  includes  an  additional  statement  about 
the  ADVICE  feature.    The  appearance  of  this  statement  under  Factor  C-II  ' 
rather  than  Factor  C-III  seems  to  indicate  the  instructor's  prominent  role 
in  English  courses  on  the  TICCIT  system:    students  viewed  instructors 
as  the  primary  source  of  the  individual,  attention  given  to  them  and  con- 
sidered ADVICE  to  be  another  system  feature.    On  the  other  hand,  students  in 
mathematics  courses  on  the  TICCIT  system  in  the  second-half  of  the  1975-76 
academic  year  tended  to  react  to  their  courses  as  a  direct  reflection  on  the 
TICCIT  program.    Their  personal  satisfaction  with  a  course  was  closely 
related  .to  their  reactions  to  specific  system  features:    Factor  B-I  reflects 
statements  on  both  of  these  dimensions  of  student  attitudes,  and  Factor  B-II 
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suggests  that  students  attributed  individual  attenticto  to  the  computer 
system's  ACVICE  as  well  as  its  oul**e  mastery  tests.    Stutfeflt*  in  these 
mathematics  courses  relied  heavily  <>n  the  TICCIT  program        viewed  their 
courses  very  much  as  a  function  tji  specific  system  features*    System  Fea- 
tures was  the  third  scale  studied       our  analyses  of  stude**  attitudes. 

The  scales  that  define  student  attitudes  for  the  purposes  of  our 

A 

evaluation  were  constructed  in  accordance  vith  the  dimensions  revealed  by 
factor  analysis*    Students'  responds  to  statements  posed  in  both  TICCIT 
classes  and  lecture  sections  of  *  coUrae  gives  us  a  basis  for  determining 
the  affective  impact  of  the  TIQCit  Program.    Beyond  the  oye*aU  effect 
evident  from  a  total  score  on  such  atatetoents,  we  will  examine  student 
attitudes  in  terms  of  key  dimensions:    Coutse  Satisfaction,  Individual 
Attention,  and  System  Features.    fteliebUity  estimates  fo*r  these  scales 
appear  in  Chapter  2:    Methods  and  Measurement ,  and  a  complete  list  of  the 
statements  included  in  each  scale  can  be  found  in  Append!*        Whatever  the 
source  or  size  of  the  estimated  treatment  effects  on  student  attitudes,  it 
should  be  kept  in  mind  that  the  effects  apply  to  those  students  who  com- 
pleted courses  on  the  TICCIT  syatenu 
6.2    Presentation  of  Results 

Descriptive  statistics  for  student  ac0res  on  the  attitude  survey^ 
appear  in  Appendix  Q.    These  sui^ttes  report  the  number  ot  observations 

i 

within  each  section  of  a  course,  mean  scores  on  all  attitadinal  statements, 
mean  scores  on  those  statements  pofsd  in  both  the  TICCIT  and  lecture 
conditions,  and  the  standard  deviations  of  these  scores.    Th*  appendix 
gives  these  figures  for  each  se^tiOtTof  a  target  course  *a  *sll  as  the 
totals  for  all  sections  within  *n  academic  term  broken  do*rn  by  instructional 
condition* 
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Two  gene^A^  Patterns  emerge  from  these  data  for  sections  of  mathematics 
courses.    Acrty0  <ioutsas  and  academic  terms  vit  seems  that  students  in  TICCIT 
classes  for  m^enfcttc* ,  when  offered  with  a  high  student-teacher  ratio  as 
had  been  the  at  Ptiosnix  College,  had  less  positive  attitudes  toward 

their  courses  £l>*n  did  students  in  traditional  lecture  classes.  Those 
sections  with         lowest  mean  scores  on  common  statements  were  usually  the 
classes  on  th«  < *CciT  system.    Yet  TICCIT  classes  also  showed  the  highest 
standard  devi^i*>n8>  suggesting  a  wider  range  of  student  reactions  and 
greater  diffeiyx^e,  among  individual  students.    This  is  almost  the  exact 
opposite  of  t*V  5>attern  of  achievement  results  in  these  same  courses.  On 
that  component       student  performance  the  highest  means  and  lowest  standard 
deviation  had  ^c^red  in  TICCIT  classes. 

A  second  ^^t^m  was  evident  from  the  attitude  results  in  a  mathematics 
course  that  ha<j,>«*n  offered  under  three  conditions.    Those  sections  of 
Algebra  I  co^yji^  instruction  had  the  highest  mean  scores 

**°n8  al &  The  mean  score  on  statements  common  across  conditions 
was  higher  in  ^£tt<W  with  programmed  instruction  than  in  classes  on  the 
TICCIT  system  &  in  lecture  sections.    But  these  attitude  scores  include 
every  responded  to  the  S^yey  regardless  of  that  student's  final  grade.  Of 
an  initial  enr^m^t  of  161  students  in  sections  under  programmed  instruc- 
tion there  wer^  ju*t  3X  students  who  completed  the  course  requirements  for 
Al^bra  I  invtt\£   1575-76  academic  year.    It  had  been  our  intention  to  re- 
strict  comparing*  to  tjtose  students  who  completed  a  course.    Otherwise  it 
would  be  diffi^t  to  specify  the  student  population  to  which  the  estimated 
treatment  ef fe<^*  Applied  and  to  collect  appropriate  attitude  data.  Responses 
from  students  V  ^cture  sections,  then,  seemed  a  much  better  standard  for 
gauging  the  attf C^ie  results  of  the  TICCIT  progranj.  # 
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There  was  no  consistent  pattern  of  attitude  results  among  sections  of 
English  courses.    Sometimes  the  highest  mean  score  on  common  statements 
occurred  in  a  section  on  the  TICCIT' system.    Often  the  lowest  mean  score  for 
the  same  course  and  academic  term  also  occurred  in  another  section  on  the 
TICCIT  systems    Extreme  section  averages  were  as  frequent  in  lecture- 
discussion  sections.     In  themselves  the  attitude  results  across  course 
sections  illustrate  the  substantial  variation  in  student  reactions  from 
section  to  section.    A  full  standard  deviation  separated  the  highest  and 
lowest  mean  scores  in  both  mathematics  and  English  courses. 

6. 2. a    Common  Statements 

The  student  surveys  contained  over  20  statements  phrased  in  an 

identical  or  closely  parallel  manner  across  instructional  conditions.  There 

were  other  statements  on  the  survey  specific  to  a  particular  condition  or 

relevant  to  student  activities*    But  the  statements  that  appeared  regardless 

of  condition  formed  the  basis  for  our  comparisons  of  attitude  results.  Most 

of  these  statements  had  an  obvious  direction,  so  stronger  agreement  with  a 

positive  item  stem  or  stronger  disagreement  with  a  negative  item  stem  t 

Clearly  implied  an  increasingly  positive  student  reaction.  Reversing 

responses  to  negative  stems  enabled  us  to  calculate  an  attitude  score  on 

J  * 

common  statements.    There  were  also  four  statements  on  eqch  form  of  the 

student  survey  that  had  no  definite  direction.    These  item  stems  solicited 

i 

students'  opinions  about  the  pace,  difficulty  and  work  load  of  a  course. 

Scores.    Tables  6.2  and  6.3  present  the  attitude  scordt  on  common 
statements.    Of  the  eleven  cases  presented  in  Table  6.2  there  were  ntae 
instances  in  which  student  attitudes  toward  mathematics  courses  Underlie 
TICCIT  program  were  les^Jpositive  than  the  reactions  given  in  lecture 


sectidfcs.    Furthermore,  these  lower  attitude  scores  occurred  across  academic 
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Attitude  Summary 
Mathematics  Courses 
Academic  Year  1975-76 


Collage 

Course 

Term 

Comparison 

Group 

N 

Number  of 
Items 

Mean 

sd 

Phoenix 

Math  007 

Fall 

TICCIT 
Lecture 

,  27 
134 

18 
18 

64.07 
69.78  • 

14.5 
9.8 

Spring 

TICCIT 
Lecture 

20 
121 

19 
19 

67.60 
74.31 

16.2 
11.3 

Math  106 

Fall 

TICCIT 
Lecture 

20 
73 

18 
18 

63.10 
71.05 

12.1 
9.1 

Spring 

TICCIT 
Lecture 

1  1 

J.  J. 

81 

1  Q 

19 

68. 36 
74.56 

12.1 
9.6 

Math  108 

Fall 

TICCIT 
Lecture 

11 

JO 

18 

1  ft 

63.82 
74.00 

14.9 
9.0 

• 

Spring 

TICCIT 
Lecture 

15 

37 

19 
19 

69.40 
76.84 

12.5 
8.1 

Alexandria 

Math  31 

Fall  , 

TICCIT 
Programmed 

21 

Li 

18 

18, 

18 

71.59 
75.05 
71.64 

11.2 
10.4 
10.4 

Winter 

TICCIT 

Programmed 

Lecture 

64 
36 

23 

19 
19 
19 

76.56 
80.89 
74.26 

10.2 
9.0 
10.1 

Spring 

TICCIT 

Programmed 

Lecture 

24 
38 
12 

19 
19 
19 

71.17 
80.21 
73*75 

1  12.5 
9.8 
9.6 

Math  32 

Winter 

TICCIT 
Lecture 

22 
20 

19 
19 

66.45 
^6.90 

12.5 
13.6 

Spring 

TICCIT 
Lecture 

18 

30 

19 
19 

71.44 
76.23 

8.6 
9.4 

Table  6.2 
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Attltude  Summary 
English  Courses 
Calendar  Year  1976 


Comparison 

N 

s% 

Number  of 

ad 

Phoenix 

.  English  19 

Spring 

TICCIT 
Lecture 

^15 
A9 

V 

21 
21 

87.80 
83.12 

9.0 
12.7 

i 

Fall 

TICCIX 
Lecture 

73 
130 

21 
21 

86.33 
80. A2 

10.1 
12.2 

1 

English  29 

Spring 

TICCIT 
Lecture 

27 

31  ' 

21 
21 

85. 7A 
85.55 

9.3 
11.9 

Fall 

TICCIT 
Lecture 

37 
A8 

21 
21 

8A.59 
81.38 

8.8 
11.9 

Alexandria 

English  111 

.Wtnfrer^ 

/  '? 

TICCIT 
Lecture 

56 
5A 

' '  21 

7A.66 
80.17 

13.6 
12.1 

\  \ 
j  Spring, 

TICCIT 
Lecture 

81 
161 

21 
21 

78. 8A 
83.01 

11.7 
11.6 

Fall 

TICCIT 
Lecture 

117 
75 

21 

73.86 
82.31 . 

12.1 
12.6 

) 


V 

0  / 


Table  6.3 
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terms  in  four  math  courses  and  only  one  oath  course  showed  mixed  results  across 
t*r«6.    Such  replications  of  attitude  results  certainly  suggest  that  stud  ems 
favored  conventional  sections  over  TICCIT  clas^fr  of  mathematics  courses. 
That  the  magnitude  of  the  difference  In  mean  oSs  was  most  often  greater 
than  one-half  of  a  standard  deviation  tends  to  support  this  interpretation. 
But  descriptive  statistics  do  not  prove  the  veracity  of  any  conclusion. 
Since  students  who  completed  mathematics  courses  under  the  TICCIT  program 
also  had  quite  different  pretest  scores  when  compared  to  those  of  students 
from  lecture  sections,  the  disparity  in ^attitude  scores  may  be  attributed  to 
dissimilarities  in  the  comparison  groups^  A  better  estimate  of  treatment 
effects  on  student  attitudes  will  come  from  an  analysis  which  takes  student 
similarity  into  account. 

The  attitude  results  in  English  courses  suggest  both  positive  and 
negative  TICCIT  effects.    Courses  which  r^gied  heavily  on  the  TICCIT  system 
for  instruction  in  the  fundamentals  of  English,  English  19  and  English  29, 
show  higher  attitude  scores  among  students  in  sections  that  spent  consider- 
^le  amounts  of  time  on  the  TICCIT  system.     The  opposite  directions  of  these 
differences  hardly  contradict  each  other.    They  instead  indicate  that  dif- 
ferences in  a  course's  coverage,  in  the  manner  of  applying  the  TICCIT 
program,  or  perhaps  in  the  colleges  themselves  can  exert  a  strong  influx** 
on  the  nature  of  impact  on  student  attitu 

Item  Responses.    Where  attitude  scores  suggest  that  students'  reactions 
to  a  course  depended  on  its  instructional  condition,  responses  to  individual 
statements  may  reveal  what  led  atudents  to  react  less  favorably  or  more 
positively  to  a  course  under  the  TICCIT^program.     The  TICCIT  program  seemed 
to  result  in  less  favorable  attitudes  in  most  mathematics  courses  and  in 
more  positive  attitudes  in  certain  English  courses.     Tables  6. A  and  6.5 
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detail  student  response  to  individual  statements  on  the  attitude  surrey  in 
order  to  Illustrate  these  apparent  negative  and  positive  effects*    Appendix  & 
contains  summaries  of  item  responses  for  all  target  courses  in  the  project. 

Table  6.4  has  response  means  and  standard  deviations  for  mathematics 
courses.    This  particular  academic  term,  the  fall  semester  of  the  1975-76 
academic  year,  marked  the  start  of  the  demonstration  phase  of  the  project. 
These  courses  involved  the  use  of  the  TICCIT  program  as  the  primary  in- 
structional resource  available  to  students;  student-teacher  ratios  in  TICCIT 
classes  were. above  those  in  traditional  lecture  sections*    By  themselves 
the  mean  responses  to  statements  may  appear  to  be  positive  and  representa- 
tive of  a  favorable  attitude:     Students  tended  to  agree  with  those  item 
stems  phrased  in  a  positive  manner  and  to  disagree  with  statements  worded  in 
a  negative  direction.     Students  who  completed  math  courses  under  the  TICCIT 
program  did  not  react  negatively  to  their  courses  but  their  attitudes  were 
less  positive  than  those  expressed  by  students  from  Tecture  sections. 
Nearly  every  statement  shows  a  less  positive  mean  response  among  students 
from  the  TICCIT  clamftoi  when  compared  against  the  mean  response  among 
students  from  lecture  sections. 

Table  6.5  contains  attitude  data  drawn  from  English  courses.  Thes* 
responses  came  from  students  in  courses  offered  in  the  last  academic  term  of 
the  demonstration,  the  fall  term  of  the  1967-77  academic  year,  and  these 
courses  had  heavier  instructor  involvement  in  those  sections  in  the  TICCIT 
system.     The  subject  matter  perhaps  required  greater  teacher  participation; 
it  also  had  a  less  formal  structure  than  mathematics.     Furthermore,  the 
TICCIT  curriculum  in  English  had  been  developed  after  mathematics  and  so 
reflected  some  experience  in  courseware  production.    Again  students  tended 
to  react  favorably  to  their  courses  in  that  they  agreed  with  the  item 
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/       Student  Attitude  Surveys 
Item  Responses  ecrosslnstructional  Condition 
Phoentof  College- 
Math  007,  106,  and  108     •  * 
Fall  Semester,  1975     '  • 

'    •  HCCIT 
.-7     '         .    .^64)  .r 


Lectu|e': 
(N  *  260) 


•  0 

Item  Steas 

Satisfaction  subscore  (6  items) : 

•course  to  my  friends, 
ms  satisfied  with  lypersonal  progress  in 
'  this  course,  .  \  , 

Initructlon  in  this  class  met V  own  particular ' 
, -needs,  • .  ;  ,.'.■>) 

My  interest  in  math  has  been  increased  by  this 
'    course,    ;  J        •    :  , 

\    ^  aethod'of  instruction  f6r  this  course  was 
too  impersonal  for  me, 

In  this  course- 1  felt  challenged  t&do/ty 
best  work,  •••  *%  H  ;; 

Attention  subscore  (4  items) : 

1  ,^ived  a  lot  of  individual  attention  from 
.  the instructor  in  this  course. 
'  4  The  instructor  seemed  genuinely  concerned  with  - 
«y  progress.,  , 
In  this  course  I  felt  free  to  ask  questions,,       , , 
UCCIT  s  comments  on  my  work  ("ADVICE")  helped 
<  me  to  progress  through  this  course. /The  Instructor's 
coaenta  on  my  work  helped  me  to  progress  through 
.  ->.  ^this  course.  '        V  "    V  P 1 

Future  subscore.  (3  items).:  t  .  , '       '  *'°f 

.  Wng  practice  problems  ("PRACTICE")  helped main  ■ 
'learning  mathematical  rules  and  concepts, homework 
assignments  helped  me  in  learning  mathema  '  " 


Item  Number 


20 

A 
6' 
19 
18 

i  . 

16 


leal 
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3,27  1,50 
3,27  1 .25 

3.43  l)i 


19 


^16 


3.91  ,0,96 
!0  1,21 


"3,08  1.35  , 
2,58\  1.24 
3,70  1,22 


2.89  1,30 

3,40  1,21 
4,09  ( 1,07 


■  •  ■  •< 

•  f-  V  Vt.'.' 


3,99  '0.9% « 

l^>°3  ^  It 

is  .;:>8|^;  % 


3.33  1.05 


".■'.V 


»      *•».»•«  ;      10 4.25/6.85- 
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Item  Sterna 

TICCIT  "HELP"  provided  clear  expiations 
when  the  material  was  difficult , to 
understand, /The  Instructor  provided 
clear  explanations  when;  the  material 
9wae  difficult  to  understand.  ;V 

The  rule  statements  ("RULE")  tra^ICCIT 
made  mathematical ' concept?  easy  to 
learn, /The  class  lecture's  made 
mathematical  concepts  easy  to  learn, 

Other  common  attitude  atems  (5, items): 

,  Taking  tests  in  this  course  let  meikndw  whj^ 
*    or  .'not  I  really  understood  the  material. 
Examples  ("EXAMPLE")  really' did  not  show  me  how  . 
to  solve  problems, /Examples  from  tjte  textbook 
and  lectures  really  did  not  show  me  how  to 
v  'solve  problems. 

In  this  course  I  felt  responsible  for  my  own 
.  learning.         •  ' 
.  Time  passed  quickly  while  I  was  in  dabs. 
r  I  tVied  to  just  finish  the  lesBOns/father  than 
f[  *  4«arn  the  material,  f 

.•  Information  statements: 


Item  Number 
(Form  035) 


Mean  ad 


Item  Number  ■ 

(Form  040)  Mean  sd$' 


10 


11 


I  wanted  information1  that  ifould  tell  me  where  I 

. stood  in  comparison  to  other  students. 
For  me,  tjjyace  at  which  I  had  to  cover  the 

mat&gfcl  in  order  to  finish  the  course  was:/ 
^  FoiS^the  pace  at  which  the  instructor 

coMd  the  material  during  the  term  was: 
For  my  preparation  and  ability  most  of  the  work 

in  this  course  was: 
The  work  load  for  this  course,  in  relation  to  other 

courses  of  equal  credit,  was: 


27 
28 
29 


Item  Coding  (Form  035): 

Sterna  1-21) 
.  5  '*  Strongly  Agree 
4  ■  Agree 
3-*  Not  SUre 
2  *  Disagree  \ 
0  1  ■  Strongly  Disagree  ° 

ERIC, 


.{items  22-26)  (item  27) 
5  ■  Almost  Always   5*  Very  Slow 

4  ■  Frequently      4  ■  Somewhat  Slow 

3  ■  NofSure        3  ■  About  Right 

2  ■  Somewhat  Fast 
1  ■  Very  Fast 

.  Table  6.4(b) 


2  *  Occasionally 
1  -  Almost  Nfter 


1,12  idl 


3.59  1.06 


1 


*'    6       4.33  0 


4       4.16  0.86 


12 

3.59  1.26 

,  7 

4,02 

1.03 

13  ( 

^3.00  1,27 

'  8 

2,07 

1,06 

14 

4,28  0.85 

9 

3,79 

1.05 

22 

4.43  0.79 

20 

1.13 

24 

;2.14  1.33 

% 

r  i.9i 

1,07 

26",  # 

2,00  1,29 

22 

2.51 

1.3p 

(item  28):.»1i5;?.> 
5  -  Very  Di£fMs|L|^.; 

4,  ■  Somewhat  Dif icult 

3  »  About  Right' 

2  -  Somewhat  Easy,  . 
1  ■  Very  Easy 


2.76  0.75 

i 

3,05  t).90 
2.89  0.81  ' 


(item  29) 
5  ■  Much  Lighter 

4  ^lighter 

3  "  About  the  Same 

2  »  Heavier 
1  *ifuch  Heavier 


'A 
■i ' 


'FsUJsa^ttJf;  ■ 

'  nccif  ' 

(H  •  110) 

•  N  ,  . '   Item  Number 

(Fora  06B)  Hun 


Item  Sim  • 
Satisfaction  subscore: 

I  tried  to  just  finish  the  assignments  rather  than  lean  the  Material,  i 
The  method  of  instruction  in  this  course  was  too  impersonal  for  ne.  ' 
I  learned  a  lot  about  grammar  in  this  course,  ' 
I  learned  a  lot  about  composition  in  this  cdtee. 
1  would  womU  this  course  [with  TICCIT]  Way  friends. 
Instruction  in  this  class  net  my  own  particular  needs.  . 
lft  this  course  I  felt  challenged  to  do  my  best  work. 
My,  interest  in  writing  has  been  increased  by  this  course. 
Attention  subscore:  .< 

1  receives  a  lot  of  individual  attention  from  the  instructor  in  this 
course.  ,  . 

The  instructor  teemed  genuinely  concerned  with  my .progress. 
In  this  course  ^  felt  free  to  ask  questions. 

Feature  subscore:  T  , 

TICCIT  "HELP"  provided  clear  explanations  when  the  materil  was  difficult 

to  understand./  The  instructor' .provided. clear  explanations  when  the  ' 

material  was  difficult  to^fcrstand. 
The  rule  statements  ("RULE")  on  TICCIT  made  concepts  in  writta&liy  to 

learn./  ,The  class  lectures  made  concepts  In  writing  easy  WBk.  I 
Doing  practice  questions  ("PRACTICE")  helped    in  learning  r^  P 

and  concepts  iD  writing./  Homework  assignments  helped  me  i'i 

learning  rules  and  concepts, in  writing.'  v  • 

HCCIT's  comments  oft  my  work  ("ADVICE")  helped  me  to  progress -through 

this  course./  The  instructor's  comments  on  ny  work  helped  me  to 

progress  through  this  course. 

(fther  stems: 

•  i 

Taking  tests  in  this  course  let  me  know  whether  or  not  I  really  ' 
(  understood  the  material. 

EraaplesJ^EXAHPLE")  really  did  not  show  me  how  to  write  well. 


27 
21 
16 
17 
23 
6 

19 
22 


3 
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1.64 

2.36 

4.13" 

3.97 

4,15 

4.31 

4.24 

3:54  * 


,3.95 
'  4        4.33  " 

1     '  4r63" 


sd 


.90 
1.05 
.72 
.81 
.97 
.76 
-.78 
1.01 


.87 
.78 
.63, 


10     ,  4.02  1,00 

v  t 

if       4.19  ,81 


9  MM  '.71. 
8  »3.7$| 


1.13. 


12  -$8 

13  •  2.71 


.62 
1.17 


Lecture 
(N  a.  177) 

Item  Number 
(Form  06  j)  Mean 


23 
'19 
13 
14 
21 

If 
17 
2Q 


:.2 

3 
1 


6 
4 

J 

'lb 


%2 


•2.06' 
2.34 
>  4.06 
3.74 
3.97 
,  3.85 
;  3.76 
3.45 


3,54 
3.99 
4.42 


0'. 


sd 


1.14 

.99' 

.98 
1.08 
1.09 
1.01 
1.06 
1.12 


1.20 
1.05 


.  f-i, 


,4.32 
3.88 


.84 
1.00 


3.83  .91 


3.97  1.07 

"•    •  /' 


i  7      ,  4.17 
;  2.65 


,91 


1. 


U 


373 


[ten  Stems 

il  this  course  I/felt  responsible  for  my  own  learning. 
Other  students  rn  tills  section  seemed  to  like  the  course. 
I  was  satisfied  with  my  personal  progress  in  this  cpurJl. 

Time  passed  quickly  while  I  was  in  class. 

Information  statements:  ♦ 

I  wanted  intonation  that  would  tell  me  where  I  stood  in 

comparison  16  tother  students. 
For  me,  the  pace  at  which  I  had  to  cover  the  material  in 

order  to  finish  the  course  was: 
For  my  preparation  and  ability,  most  of  the  work'in 

this  course  was: 
The,  work  ImI  for  this  course,  in  relation  to  jfther  courses 

of  equal  fredlt,  was: 

Item  Coding  (Form '068): 

•  (Items  1-24) 
5  *  Strongly  Agree 
4  ■  Agree 
3  *  Not  Sure 
2  ■  Disagree 
,1  ■  Strongly  Disagree 


Items  25-29) 
5  ■  Almost  Always 
4  "^Frequently 
3  ■  Not  Sure 
2  ■  Occasionally 
1  ■  Almost  Never 


.    (Item, 30) 
5  ■ Very  Slow 
4  «  Somewhat  Slow 
3  ■  Aboat  Right 
I  «  Somewhat  FaBt 
J   Very  Fast  . 


Tree ii 

'  (H»llO) 

Item  Nunber 
(Form  068)  Mean 

4.44 
3.61 
4.22 
4.40 


20 

25 


(Item  31) 
5  ■  Very  Difficult  ■ 
4  ■  Somewhat  Difficult 
3  ■  About  Right 
2  ■  Somewhat  Easy 
1  ■  Very  Easy 


sd" 

.61 
.81 

.90 
j.  83 


29  t 

2.55 

1.38 

30 

2.90 

.65 

31 

2.79 

.76j 

32 

3*.65 

M 

Lecture 
CM  as  177) 

Itet  Number 
(Form  067V  Mean 


9 

15 

18 

22 


24 
25 


27 


3.98 
3.59. 
3.61 
3.70 


2.64 
2.94 
3.00 
3.19 


(Item  32) 
5 /Much  Lighter 
/-Lighter  * 
3  ■  'About  the  $M*M 
2  ■  Heavier  ) 
1  *^ich  Heaviej£ 


in 


v 


Table  6.5(b) 
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stems.    But  thesesdata  show  mean  responses  from  TICCIT  classes  higher, than 
those  fro.  lecture-dla«»sa*on  sections.  'Almost  all  statements^  responses 
^iPicative  of  a  more  positive  attitudes  among  students  who  had.^ompleted  the 
C    courses  on  the  , TICCIT  system  rather  than  in  a  lectiire-discussion  s^tion. 

What  mean  responses  to  individug.  statements  t«H  us  about  student  stti- 
tudes  may  be  unclear.    Yet  such  responses  can  provide  useful  information  in 
understanding  the  estimates  of  tAtment  effects  on  student  attitudes  and 
in  reaching  conclusions  about  the  strengths  and  weaknesses  of  the  TIC^T 
program.  -It  might,  therefore,  be  worthwhile  to  consider  ^least  a  fgw 
examples,    tte  first  item  listed  in  Table  6.4,  J^fc20  on  Form  03& "I  would 
recommend  this  course  to  my'  friends,"  had  mean  resftises  of  }%%f  in  TICCIT 
classes  and  3.91  in  lecture  sections.    The  response  scale  for  that  stateMnt 
went  from  strong  agreement  to  strong  disagreement,  scored  as  five  points 
down  to  one  point.    While  students  in  lecture  sections  agreed  that,  they 
would  recommend  their  mathematics  course  to  fr^pls,.  Students  completing  the 
same  course  un/eN<he  TICCIT  program  were  unsure*  about  making  such  a  recom- 
mendation., Distributions  of  item  responses  make  this  point  even  clearer- 

f 

|one-fourth  of  the  students  in  TICCIT  classes  would  apt  recommend  the  course 
while  less\than  one-tenth  of  the  students  in  lecture  sections  would  do  the 
^same.    Surely  this  has  implications  for  student  satisfaction  with  {he  in-  • 
>    structional  condition.  -  ~^ 

,  Other  response  means  from  Table  6.4  indicate  that  lecture  sections 
better  met  each  student's  particular  needs  tit em  6  oh  Form  035)  that 
V  s tudenta  received  more  individual  attention  from  instructors  in  lecture 
sections  (item  3  on  Form  035),  that  instructors  in  lecture  sections  pro-  * 
vided  much  better  explanations  ol  difficult  material  than  did  the  HELP  f 
feature  on  the  TICCIT  system  (item  10  on  Form  035).    Admittedly  these 
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responses  reflect  student  opinions/and  their  perceptions  may  be  inaccurate.^ 
They  do,  however,  reveal  how  students  felt  about  their  courses  end  remain 
pertinent  in  evaluating  impact  on  attitudes.    FTom  the  student*'  standpoint 
only  the  practice  problems  on  the  fXCCIT  system  met  the  -standards  of  lecture 

■ 

eections.    Computer-assisted  instruction  in  itself*did  not  guarantee  in- 
struction  tailored  UftMft*  individual  student. 

Most  of  these  ob^rvation\  can  be  repeated  for  the  English  courses  ' 
included  in  Table  6.5  with  the  opposite  emphasis.    Students  in  these  TICCIT 
classes  did  feel  that  the  instruction  had  better  met  their  own  particular 
n*4ft»  >rfem  6  on  Form  068),  that  thmg  had  receive*  greater  ind4*Wual 
attention  from  instructors  (item  3  om%ori  068),  that  gmmcticeTSrVblems  eh 
the  TICCIT  system  helpedigthem  in  learning  rules  and  concepts  in  writing 
(Item  9  on  Form  068) .    And  most  students  who  had  taken  these  English  courses 

on  the  TICCIT  system  would  strongly  recommend  the  same  course  to  thelr^ 

V'.  *'■  «• ' 

friends:    the  majority T6f  students  from  TICCIT  classes,  either  strongly 

agreed  (42Z)  or  agreed  (40Z)  with  this  question  on  ^  survey  (item  23  W>  :: 
Form  066).    Unlike  other  aspects  of  student  performance,  it  appears  that  the 
TICCIT  program  affected  student  attitudes  sometimes  in  a  positive  manner  and 
sometimes  in  a  negative  manner  when  compared  againat  the  reactions  of 
students  in  traditional  sections  of  the  same  courses. 
6.2.b    Specific  Statements 

The  student  surveys  contained  somS  statements  that  were  specific 
to\an  instructional  condition^Nrheae  statements  covered  aspect*  of  TICCIT  ^ 
classes((BBBlfeections  withLut  definite  parallels  in  another  condition. 
Despi^  fl         Pf  parallel  and  a  consequent  restriction  on  any  comparisons 
across  con^^Ki?      believed  that  studtot  reactions  to  these  unique  course 
attributes  should  be  obtained-    tt&M&*&  the8e  8Peclflc  •t^tementa  might 


* 


• '  ...  • 

b«  likened  to  the  role  of  the  items  on  the  achievement  poettea**  that  . 
fleeted  objectrvee  unique  to  on**onditlon  or  another:    they  b*lP  u"  **> 
documenting  outcomes  of  Importance  Independent  of  their 

frequency  o£  &c*^ 


rence.    By  the  eaae  tok^n  there  may  be  specific  eteteaents  of  **o&** 

A  set  of  Item  sterna  permits  atudent  responses  on  a  limited 

coiqponenti.    Students'/ own  comments  may  go  beyond  these  comp<»e,lt*  '****U. 

ing  out  other  instructional  strengths1  or  weaknesses  in  a  course-  Thea* 

comments  constitute  an  additional  source  of  Specific  state 

J|e  reflect!^ 

student  attitudes. 

Item  Responmms.    Table  6.6  presents  response  leans  and  st**"***^ 

ddfcfcatiotis  for  survey  statements  unique  to  a  particular  instruct**0 
condition.    The  courses  included  in  the  table  are  the  "same  as 
In  the  Item  summaries  for  common  statements  (eee  Tables  6.4  and  6»5)* 
Appendix  R  has  similar  tables  for  ell  courses. 

From  the  statements  spetelflc  to  TICCIT  dtasses  It  Is  appar«ot  that 
students  could  use  the  system's  imp  structures  to  guide  their  lea**1111*  °f 
the  subject  matter  C|tej*  7 j&Jpm  035),  and  could  readily  mast«*  ^^atj. 
leal  procedurea  necessary ''for  operating  the  terminals  (Item  23  o*  fotm  °35) . 
The  TICCIT  program  did  enable  each  student  to  set  a  pace  approprlat*  t0  hts 
or  her  ability,  and  it  made  students  active  participants)  In  the  p*oce8S  of 
learning.    There  was  a  clear  consensus  of  agreement  with  stated0  reX«V«nt 
bo  these  two  feature*  of  the  progrem  (Items  2  and  5  on  Form  jfo) •  DeBP*te 
the  short  duration1  of  aystem  f  ailuree ,  (aee^Chapter  7:    Student  A0-nti**) , 
Interruptions  of  computer  serv^e  disrupted  student  learning  (iteBI  15  °n  . 
Form  035).    The  lower  Incldeno^jf^agreement  with  this  statement  English 
courses  offered  on  the  flCCIT*  system  a  full  year  plater  than  the  matflie*itlca 
courses  seems  to  Indicate  tha^  disruptions  caused  by  computer  bre*^0*** 
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diminished  with  time.  Student  opinions  about  taKitt^nother  course  on  the- 
TICCIT  system  (item  21  on  Form  035)  and  about  needing  a  regular  textbook  as 

mmmV '  *' 1 

a  convenient  refetWe  (item  25.  on  Form  035)  varied  considerably.  The  re- 
sponse means  to  these  two  statements  'suggest  that  students  generally  Jfculd 
enrom  in  attoper  course  under  the  TlCCtT  program  and  felt  no  need  for  a 


regular  tfjflj      Wt "ftowever,  «a  sizeable  percentsge  of  students  in  mathematics 
courses  vtf^j  Jpld  subsequent  TICCIT  classes  (29X)  and  a  large  proportion 
(39%)  wi^^/i^sxtbook  to  be  made  available  for  use  away  from  the  computer  i 
system. 


t  responses  to  statements  applicable  bnly  to  lecture  sections 


V, 


offer  additional  insight  for  interpreting  the  attitude  results.  Although 
students  in  English  courses  found  their  Textbooks  useful  in  explaining  and 
presenting  materia}  (item  5  on  *orm  067).  mathematics  textbooks  were  viewed 
as  less  helpful.    Indeed.  one-hal#f  of  the  students  in  lecture  sections  of 
math  courses  did  not  find  their  textfcooks-  to*be  effective  in  explanation* 
\and  presentations  of  problem  solving.    If  the  tlCCIT  program  replaced  just 
textbooks,  the  appropriate  standard  tot  comparisons  of  attitude  results 

would  be  much  lower  in  mathematics  courses.    But  the  program  had  been  In- 

I 

tended  to  supplant  regular  classroom  instruction  and  raise  students-teacher 
ratios.    Given  students'  favorable  reactions  to  class  discussions  and  to 
student  questions  as  devices  for  learning  (it ep  11  on  Form  035).  there  was 
an  important  element  of  classroom  interactions  absent. from  TICCIT  classes. 
The  English  courses  on  the  TICCIT  system  that  incorporated  supplemental 
small  group  discussions  were  the  TICCIT  classes  which  showed  positive  atti- 
tude  reaults  in  comparisons  with  lecture-die cuss ion  sections  (see  Table  6.5) ?- 
i  Student  Comments.    The  attitude  surveys  provided  students  with  an 

'  -  '  ■  .         •*  a 

opportunity  to  comment  freely  and  to  elaborate  on  their  reactions'  to  a. 
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coursed    There  were  several  recurrent  points  in  these  comments,  mostly 
remarks  about  specific  features  of  the  TICCIT  program.^  Although  students 
often  offered  additional  praise  of  the  program,* these  positive  comments 
tended  to  be  redundant  with  their  responses  to  particular  statements  already 
on  the  survey*    It  is  the  constructive  criticism  that  seems  most  pertinent  * 
to  further  refinements  of  the  TICCIT  program  and  to  understanding  student 
attitudes  in  mathematics  courses. 

Students  frequently  raised  six  points  in  commenting  about  a  course 
under  the  TICCIT  program.    The  high  student-teacher  ratio  in  some  mathematics 
courses  led  to  complaints  about  ready  access  to  instructors.    There  was  such 
demand  for  instructor  assistance  in  explaining  concepts  and  supervising 
mastery  tests  that  individual  students  had  to  wait  for  answers  to  their 
questions  about  content.    Two  other  points  often  mentioned  by  students  begin  ■ 
to  account  for  their  need  of  instructor  assistance.    When  students  requested 
additional  help  on  the  TICCIT  system,  the  explanations  Jfiven  to  them  still 
seemed  to  skip  steps  involved  in  applying  a  rule  or  in  solving  a  problem. 
The  level  of  detail  given  ifi  the  H^LP  sequences  was  not  sufficient  for  all 
students.    Also,  students  with  a  specific  question  about  a  rule  or  problem 
turned  to  instructors  for  an  answer.     The  TICCIT  program  could  provide 
answers  kto  just  those  student  questions  that  the  courseware  developers 
had  anticipated  and  incorporated  into  the  curriculum;  the  computer  system 
could  not  respond  to  a  direct  inquiry. 

Students  also  commented  on  the  structure  o^  the  TIQCIT  curriculum  which 
required  that  students  master  one  unit  before  proceeding  to  another  unit 
higher  on  the  courseware  map.     Sometimes  this  impeded  rather  than  facili- 
tated student  progress.     Students  cited  the  fact  that  lecture  sections 
permitted  them  to  continue  with  their  studies  and  to  return  later  to  an 


omitted  topics  when  they  had  &  better  grasp  of  the  subject  matter.     A  student 
stuck  6n  a  unit  in  the  TICCIT  curriculum  had  to  stay  with  that  unit. 
Perhaps  the  inordinate  amount  of  time  spent  on  particular  units  prompted  f 
other  remarks  about  the  need  to  devote  an  excessive  amount  of  time  to  TICCIT 
classes  or  to  cover  too  much  materia^..    Although  responses  to  particular 
item  stems  had  indicated  that  the  work  load  and  pace  of  courses  under  the 
TICCIT  program  generally  matched  student  ability,  some  students  objected  to 
the  study  demands  of  TICCIT  classes.     Finally,  students  beiieved  that  the 
TICCIT  system  mistakenly  judged  their  correct  answers  to  be  incorrect. 
ft  caused  noticeable. frustration  when  the  computer  system  failed  to. recog- 
nize correct  answers  unless  the  answers  conformed  to  its  notatiotial  conven- 
tions, or  when  system  feedback  on  student  solutions  to  practice  problems 
failed  to  acknowledge  a  correct  answer.  f 

Some  of  these  comments  suggest  specific  steps  for  improving  the  TICCIT 
program.     Refinements  to  the  procedures  for  judging  answers  could  enable  the 
computer-  system  to  accommodate * equivalent  forms  of  cor^ept  answers  for 
practice  problems.     The  hierarchical  structure  could  b^  made  flexible  enough 
for  students  to  leave  a  particularly  troublesome  unit,  especially  if  it  were 
not  essential  to  other  topics,  or  the  curriculum  might  attempt  alternative 
explanations  of  difficult  concepts.     Course  requirements  might  he  adapted  to 
reflect  realistic  expectations  of  what  students  could  accomplish  within  an 
academic  term,  or  colleges  might  award  partial  credits  in  recognition  of  the 
progress  that  each  student  had  made  within  a  term.     And  the  student-teacher 
ratio  should  not  exceed  the  usual  ratio  of  regular  classes  unless  experience 
warrants  a  higher  load. 

Yet  it  would  seem  impractical  to  attempt  resolving  all  student  criti- 
cism.    The  resources  necessary  for  developing  a  computer  system  capable  of 
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,  responding  to  any  student  question  about  the  subject  matter  would,  at  the 

.present  st^e  "of  instructional  technology,,  be  prohibitive.     Similarlly,  it 

\ 


may  De  impossibre  -to  construct  remedial  instructional  sentiences  of  suf f i-  v 
cient  detail  to  satisfy^ every  student  in  as  heterogeneous  a  population  as 
fc-ound  at  community  colleges.    /The  instructor  has  an  essential  role  in 
*     applications  of  computer-assisted  instruction.  > 

These  comments  about  the  TICCIT  program  had  been  chosen,  for  their  poten- 

,j  '  . '  *-  * 
jdal  usefulness  in  stibadqufent  implementations..   There  were  also  frequent 

'  i  >  *  ^  •  -.  ,' 

positive  remarkls  abojlt  ,£ofh'  MCCIT  classes^and  lecture  sections.     It 'would 

■  '    '(   v.  ■  •  ■  "   '  '    '  * 

%e  inappropriate  to  Infer'- that,  students  were  always  or  mostly  critical  df 
t       %     ■*.  «    .  j 

«*  \  ■'  '  ' 

the  TICCIT  system.     Each  instructional  condition  had  definite  strengths.  As 

•  -  "  '  ■  ■  -  !  c 

a  student  in  a  mathematics  course  under  the  TI£Clt  program  wrote: 

The  amount  of  *  knowledge  I  have  acquired  and  the  speed  at  which  I 
^)  had'  done  so  would  almost  certainly  have  been  impossible  through 

more  conventional  instruction,   i.te.  ,  lecliire-textboo> 

\  '>  .  j 

Another  student's  comment  about  a  lecture  section  of  a  Inat  Hematics  course 

i  "     *  ;%  ^  ■' 

•was:'  '  />  ^  >\t 

v      >^  .  ,  ,  i     ■        .    ..  » 

The  instructor  is  marvelous.     Sh\  knows- her  ^matertaa  and  is  in- 
terested in  it  and  made  a  great  ejEfort^to  get  us  interested  in 
it  and  math  in  general.     Her  approach  really  anc}  hqnestly  got  me 
interested  in  the  subject  and  it  changed  my  view  of  math.  flow, 
I  like -math.  v-  '  "  '  " 

Perhaps  the  English  classe^on  the  TICCIT  system  that  incorporated  occasional 
small  group  distTTssions  combined  the  strengths  ,bf  both  approaches  arW  there- 
fore had  a  posative  impact  on  student  attitudes. 
§,3°  Analysis  of  Program  Effects  ' 

The  differences  in  student  attitude  scores  evident  from  the  descriptive  j 
data  may  reflect  real  treatment  effects  or  other  factors.     If  students  of 
higher  entrance  ability  tended  to  react  less ) favorably  to  their  courses,  the 
higher  pretest  scores  of  students  completing  mathematics  courses  under  the 
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TICCIT  program  may  account  for  some  of  the  difference .     Perhaps  the  in- 

stjructors  responsible  for  English  courses  on  the  'TICCIT-  system,  especially  r 
......  J..  >  ' 

courses  with  regular  classroom  sessions  as  well  as  computer  classes,  were 

particularly  energetic  and  enthusiastic  or  particularly  unimaginative^  and 

v     y  ^  >  :  -  ' 

dull,  and  these  difference^  anjojgg  teachers  might  explain  apparent  treatment 

effects.     Or  the  differences  in  attitude  scores  may  have  been  too  small  to 

-w       -  "  < 

justify  drawing  conclusions  about  the  affective  impact  of  the  TICCIT  program* 

Our  approach  to  data  analysis  allows  us  to  isolate  ^eatment  effects 

and  consider  other  plausible  explanations  for  observed  differences  in 

student  attitudes.     Again  the  number  of  student^  in  the  population"  of" 

interest  ,^  students^  who  completed  courses  under  the  TICCIT  program,  will 

restrict  the^set  of  independent  variables  that  san  enter  into  -djve ^analyses 

for  math  courses.     There  were  too  few  observations  from  this  population  to 

use  a  full  set  of  demographic  characteristics  in  defining  student  similarity. 

As  in  the  analyses  for  student  achievement,  we  will  rely  on  entrance  ability 

in  mathematics  to  identify  students  from  lecture  sections  comparable  to 

those  stutients  who  completed  math  courses  on  the  TICCIT  system.     In  English  , 

courses  the*^  were  higher  completion  rates  and  larger  sample  sizes.  The 

analyses  for  English  courses  can,  therefore,  take  into  account  numerous  pos- 

/     4  Vi- 
sible sources  of  differences  in  student  att/itudefe  besides  student  attitudes. 

/ 

The  estimates  of  treatment  ef  f  ects$  pres*jen>ted  here  reflect  the  influence 

*  M  ) 

of  the  TICCIT  program  on  student  attitudes.     Each  estimate  represents  the  dif 

ference  between  the  actual  attitude  score  ?>6f  a  student  who  completed  a  coufse 

under  the  TICCIT  program  and  the  probable  score  of  that  student dhad  he  or  she 

•  .  V  1      ■  4 

been  in  a  lecture  section  of  the  same  course.     We  estimate  treatment  effects 
on  the  total  attitude  score' for  all  cpinmon  statements  as  well  as'  on  subs.cores 

.    /  *         •  M  ,.- 

for  the  scales  established  through  factor 'analyses /of  item  responses. 


6. 3. a    Matfr  Courses 

Tablgfc  6.7  present  the  analyses  of  attitude  results  for  ^ 

math  courses,    ipf  the  12  aiwrly^s  for  courses  within  *ewns,  tjie  direction  of 

the  estimated, treatment  effects  was  negative  in  11  cases.     Students  reacted 

•  y  ■  ■  ] 

less  favor^b^  to  a  mathematics  course  under  the  TICCIT  program  than  to  the 

pame  cpurse^ 'in  a  lecture  tormat.-  -Yet  ther«vwere  only  thr^e  instances  in 

which  the  fmpact  on  students '/  overall  attitudes  toward  a  course  reached  a 

statistically  significant  level.    Furthermore,  tfrfer^was  just  oue  case  of  a 

significant  effect  on  the  attitude  subscore  representing,  students'  personal 

satisfaction  with  their  course;    But  the  difference  in  students'  reactions 
* 

to  specific  features  of  the  TICCIT  system  and  their  regions  to  the  coun- 
terparts  gf  Xhas%  features  in  sections  was  'signif leant  in  ten  analyses: 
students  consistently  rated  the  basic  options  of  TICCIT's  learner  control 
as  less  helpful  in  their  learning  than  the  parallels  of  these  options  in 

lecture  sections. 

The^three  cases  of  a  statistically  significant  negative  effect  on 
overall  attitudes  occurred  in  mtA  courses  which  had  higher  student -teacher 
ratios  in  TICCIT  classes  than  in  lecture  sections.    Moreover,  every  esti- 
nated  treatment  effect  was  negative  in  such  courses.     Tablfe  6.7  shoWs^ these 
results.     In  this  table,  as  well  as  in  those  which  follow,  the  regression 
weight  (b)  for  the  independent  variable  representing  instructional  condition 
corresponds  to  the  estimated  treatment  effect  (t )  and  the  value  of  the 
t -statistic  (t)  associated  withx this  variable  indicates  the  statistical 
significance  of  the  effect.     Significant  differences  in  overall  attitudes 

(i.e*. ,  students'  scores  on  those  statements;  that  appeared  in  identical  or 

A 

closely  parallel  form  across  conditions)  can  be  seen  in  Math-  007  for  the 
"fall  -and  spring  semesters  and  in  Math  108  for  the  spring  semester  of  the 
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Conetant 


Condition 

TICCIT(1)/Lecture(0) 

Entrance  Ability 
Pretest 

Conetant 


Condlt  ion 

TICCIT(,1)/Lecture(0) 

Entrance  Ability 
Pretest 

Constant 


Condition 

TICCIT(1)/Lecture(0) 

Divieion 

Day(l)/Evening(0) 

Entrance  Ability 
Pretest 

Constant 


-3.97^-1.03      -  .  24    -  v14 


.08  .21 
.73.  55  ^ 
R  -  .12 


.05  .32 

22.09       *  \ 

I 

R  -  .04 


-2.14  -1.98 


.04     -  .36 


1^.77 


.24 


-2.29  -7.16*- 

-06  4  .64 
16.07  1 
R  -  .24 


-7.39    -1.91      -2.34    -1.29      -3.59  -3.42* 

%  • 

-  .15    -1.00      -  .05    -  .69      -  .03  -  .62 

"81.26  26.44  17.69  ' 

R  Z  -36  R  -'\Z5  R  -  .51 


-1.73  -2.70* 
-  .03    -i.o£  ' 

14 .90  ' 
 R  -  .46 


2.66  -3.53** 


-7.42    -^0  -2.08    -1.19  ^,-2.42^  -2.01 

9.  95      2.  90**  2  .  96      2.03*  2.>29^2.29* 

-  .43    -  .99  -  .lo  j*  .56"     f*13  -1.01 

71. jo  .  "21.73  14.93 
*  R  -  .41 


-2.43  -2.29* 
2.24  2. 


R  -  .3^  R  -  .38  - 


-   .C9     -  .79 
16.30  I 
R  r  .41 


p<.05 
%<.01 


Table  6.7 

'    3&Z  :;: 


■  V       j     J  > 
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Attitude  Results:    Analysis  on  Co— on  Statements 
Mathamatlcs  Courses,  Alsxandria 
Acsdsmlc  Year  1975-76. 


S 


bsr  of 


/  Nujbi 
Course/    Term     .  Students 


Independent 
Variable 


Dependent  Variable 


Overall 
Attitudes 


Course 
Satisfaction 


Individual 
Attention 


Syatea 
Featurae 


Math  31  Fall 


38 


Condition 


/ 


Winter 


61 


#TICCIT(1)/Lecture(0) 

-5. 

59 
* 

-1.17 

-1.99  - 

.89 

-  .45  - 

,-2.36 

-2.76** 

Entrance  Ability 
Pretest 

15 

.85 

..06. 

•  .68 

.00 

H 

.00 

.06 

1.89 

Constant 

68 

44 

21.  ir 

^7.44 

10.58 

R 

-  .21 

R  - 

.16 

jT  -      R  ■ 

.08 

R 

-  .44 

Condition 

TICCIT(1)/Lecture(0) 

J 
1 

.21 

.40 

V* 

.15 

.11 

1.10  . 

1.33 

-  .60 

-  .68 

Jf     /  Division 

'    /  Day(l)/Evening(0) 

I  .Entrance   Ability  ' 
I  'TfStest 


Spring 


29 


iMath  32  Winter 


32 


Constant 


Condition 

~  TICCItf(l)/Lecture(0) 

Division 

Day(l)/Evening(0) 

i 

Entrance  Ability 
^Pretest 

s 

Constant  * 


Condition  V 
TICCIT(1)/Lecture(0) 

I 

Division 

,Bay(l) /Evening  (0) 

Entrance  Ability,. 
Pretest  m 


-5.57  -2.12* 


.19  -1.52 


$2^36 


.32 


-2.^T"*^2.0j* 

L  >>\ 
-  .07    -1.2£  'm-/f 

25.74     ^  ^1. 86 


.91  -1.28v 


R  -  .25 


-1.32  -1.74 

-  .05  -1.51 
18.16 

R  «  .28 


5J8/^-1.02 


^/.57    -1.06       -3.08  ^2.87**      -  .01    -  .01 


58 


\*~  Constant 


A5    v#  Condition 

TlCCIT(^)/LectureC0^ 


Division 

Day(l>/Evening(0) 

Entrance  Ability 
Pretest 

Constant  % 


3,02 
% 
\0\    -  .15 


76.69  , 


1.83        .82^        1.45  1.46 

.^1  .14  -  .05  -1.43 
22. d$      \  19.64 

R  -  £y                 R  m  ^ 


-1.00    -  .78 

.06  1.35 
13.08 

R  -  .35 


-7  .  941    *?.  02 

\ 

-14.39  -3.84*** 

f 

t  .15     -  .29 
t  85^53 j 

R  m  .66 


-2.62  -1.73  yf.23  -1.19  -3.71  -3.52** 
-5.24     -^63**    -3.22    -3.29**      -2.41  -2.40* 


-  .18 
28.37 
R 


.86 


.64 


-  .01  -  .05 
18.29 

R  -  .57 


.02  .16 
17.52 

R  -  .65 


i 


-A. 33  -1.47 

-5.73  -2.06* 

-  ,35  -  .70 
8A.58 

R  -  .A3 


-1.56  -1.19 


-2*68  -2 


^17* 


.12  .15 


.88  -1.18 


-  .25     -1.16       -  .04    -  .30 


28.44 
R 


17.96 

\ 


-  .44 


R  -  .18 


-3.22  -3.96*** 

.20  .26 

-  .04    -  .26 
16.82 

R  -  .56  „ 


 r\ 

*p<  .'05 
** 

p<.0l 

*** 

.  ?< .001 


4 


7 


^Includes  tfnly  those  students  who  received  course  creditN 
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1975-76  academic  year.     The  ^ize  of  these  effects  and  most  others  listed  in 
Table  6.7  is  roughly  one-half  of  a  standard  deviation.     This  would  be  large 
enough  to  change  the  mean  responses  on  six  or  seven  statements  from  a 
consensus  of  agreement  with  a  positive  item  stem  to  an  uncertain  position. 
Clearly  the  TICCIT  program  had  a  negative  impact  on  student  attitudes  toward 
mathematics  courses • 

Looking  at  the  estimated  treatment  effects  for  attitude  subscores  we 
can  identify  the  source  of  this  negative  effect.     Since  there  was  oqly  one 
significant  effect  on  the  scale  of  Course  Satisfaction,  it  would  seem  that 
students  viewed  the  course  itself  and  the  fit  of  instruction  in  the  course 
to  their  own  needs  as  favorably  in  TICCIT  classes  as  in  lecture  sections. 
Math  courses  under  the  TICCIT  program  generally  met  the  standards  of  tradi- 
tional lecture  practices  if  we  use  students'  personal  satisfaction  with 
a  course  as  a  criterion.     However,  the  scales  for  Individual  Attention  and 
System  Featutes  reveal  less  desirable  outcomes. 

Two  of  the"  courses  included  in  Table  6.7,  specifically  Math  (507  and 
Math  108,  show  a  statistically  significant  treatment  effect  on  the  attitude 
subscore  for  individual  Attention.     The  effect  was  evident  in  both  the  fall 
and  spring  academic  terms*     Furthermore,  the  same  attitude  subscale  had  a 
negative  estimate  for  treatment  effect  in  each  course  where  the  student- 
teacher  ratio  in  TICCIXfclasses  exceeded  that  in  lecture  sections  (i.e., 
every  cource  in  Table  6.7).     The  cbflftclusion  seevs  clear:     Increases  in  the 
student-teacher- ratio  for  math  courses  under  the  TICCIT  program  will  detri- 
mentally affect  student  perceptions  oft  the  amq^t  of  individual  attention 
devoted  to  them.    This  happened  especially  in  those  courses  which  consti- 
tuted the  first  math  course  taken  by  community  college  students,  and  it 
happened  despite  the  availability  of  proctors  for  assisting  students. 
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The  analyses  for  attitude  results  also  led  to  statistically  significant 
estimates  <^f  treatment  effects  for  every  mathematics,  course  on  the  attftude 
scale  of  System  Features.     Each  estimated  effect  on  this  sca^e  was  negative 
and  significant  at  Phoenix;  the  effect  held  across  courses  and  terms*  At 
Alexandria  there  was  a  significant ,  negative  effect  in  both  Algebra  I  and 
Algebra  II  (Math  31  and  Math  32),  and  the  effect  was  replicated  in  Algebra 
II.    Of  the  12  analyses  performed  for  examining  attitude  results  in  math 
courses  there  were  ten  cases  of  a  significant,  negative  effect  on  the 
attitude  scale  of  System  Features.     The  functions  called  RULE  and  HELP  on 
the  TICCIT  system,  intended  to  present'  mathematical  rules  and  concepts  and 
to  provide  assistance  in  understanding  rules  and  solving  problems,  received 
less  favorable  student  ratings  than  did  class  lectures  and  instructor  ex- 
planations.     Yet  RULE  statements  served  much  the  same  purpose  as  lectures 
in  that  they  both  present  material  in  an  expository  manner,  and  HELP  se- 
quences were  essentially  replacements  for  the  elaboration  an  instructor 
might  give  on  difficult  material.     With  the  notable  exception  of  practice 
problems,  which  seemed  to  fulfill  the  role  of  homework  assignments  (see 
responses  to  item  9  in  Appendix  R),  students  believed  traditional  practices 
to  be  of  greater  assistance  in  their  learning  than  particular  features  built 
into  the  TICCIT  system. 

The  pattern  of  attitude  results  in  mathematics  courses  is  as  clear  and 
consistent  as  it  had  been  for  course  completion  rates  and  student  achieve- 
ment.    But  the  estimated  treatment  effects  on  course  completion  rates  had 
been  negative  and  significant,  and  those  for  student  achievement  positive 
and  significant.     Estimates  of  treatment  effects  on  overall  student  attitude 
were  negative  and  generally  not  statistically  significant.     Instead  there 
were  statistically  significant  differences  on  particular  attitude  scales. 
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In  courses  with  a  higher  student-teacher  ratio  in  the  TICCIT  condition  than 
in  lectute  sections,  students  reported  receiving  less  individual  attention 
when  a  mathematics  course  fell  under  the  TICCIT  program.     Throughout  math 
courses  students  rated  specific  features  of  the  TICCIT  system  less  favorably 
than  the  aspects  of  lecture  classes  corresponding  to  those  features. 

6.3.b    English  Courses 

Tables  6.9  and  6.10  present  analyses  of  attitude  results  in  English 
courses.     These  analyses  reflect  the  same  approach  to  estimating  treatment 
effects  as  had  been  applied  for  mathematics  courses.     However,   the  popula- 
(vffin  of  students  completing  a  math  course  under  the  TICCIT  program  was 
generally  limited  in  number.     The  larger  sample  of  such  students  in  English 
courses  permitted  the  use  of  a  fuller  set  of  independent  variables.  Fur- 
thermore, students  in  English  courses  responded  to  the  same  form  of  the 
attitude  survey  across  academic  terms.     This  enabled  us  to  collapse  attitude 
data  across  terms  and  conduct  one  analysis  with  an  even  larger  number  of 
students  in  the  relevant  population.     In  English  courses  we  rely  on  an  ex- 
panded set  of  variables  for  identifying  similar  students  and  a  larger  number 
of  observations  for  determining  possible  treatment  effects. 

The  analyses  for  English  courses  conducted  with  small  group  discussion 

as  a  supplement  %o  instruction  given  on  the  TICCIT  system  appear  in  Table 

6.9*     Since  interaction  terras  entered  into  the  regression  model,   the  regstes- 

sion  weight   (b)  for  the  variable  representing  instructional  condition  does 

not  correspond  to  an  estimated  treatment  effect.     Instead  the  sum  of  that 

coefficient  and  the  product  of  the  interaction  term  yield  an  estimate  of  the 

impact  of  the  TICCIT  program  on  student  attitudes.     Since  the  mean  age  of 

students  in  TICCIT  classes  was  just  over  23  years,  we  can  calculate  the  • 

average  treatment  effect  on  overall  attitudes   (OA)  as  t      -  (15.2457)  + 

OA 
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Attitud€  Results:    Analysis  on  Common  Statements 
English  Courses,  Phoenix 
Calendar  Year  1976 


Dependent  Variable: 

Number  of  Students: 
Multiple  Correlation: 


Overall 
Attitudes 

316 

.34 

b  t 


Course 
Satisfaction 

316 

.32 

b  t 


Individual 
Attention 

316 

.36 

b  t 


System 
Features 

316 

:29 


Condition 

TICC1T ( 1) /Lec tur e ( 0 ) 

Entrance  Ability 
Essay  Pretest 
Objective  Pretest 

Division 

Day (1) /Evening (0) 


Term 
Spr 


ing(l)/Fall(0) 


Instructor 
II 

13 

Student  Status 

Full-time (1) /Part-time (0) 

Enrollment  in  Course: 

Repetition  of  course(l)/ 
First  enrollment (0) 

Employment 

Full-time/  3) /Part-time(2) / 
None(l) 

Maturity 
Age(years) 

Sex 

Male (1) /Female (0) 

Other 

TICCIT*  Age 
TICCIT*  Sex 

Constant 


15.25      3.08**        7.92     3.52***  .98     3.41***  .88  1.72 


1.00  .96 
■  .22  -1.91 


4.34 
-3.72 


1.71 
-1.05 


3.77  .82 


.27  2.42* 


-2.31  -1.67 


84.87 


.43  .91 
-  .11  -2.15* 


-1.94     -  .91  -  .30?-  .31 


3.59      2.27*  1.05  1.46 


1.42  1.23 
-1.44  -  .90 


-1.47     -  .80  -1.40  -1.68 


1.28  .62 


-1.99    -1.95  -  .92  -1.99* 


.14  2.74** 


.81  -1/28 


,47     -2.28*        -  .27  -2.95** 


32.45 


,19  .99 
,01  -  .21 


.44  -1.10 


95  3,23** 


.96  2.01* 
.09  -  .14 


,42  1.24 


94  1.09 


-  .35  -1.84 


.04  2.26* 


,09  .34 


<2k 

11.24 


.15  .59 
.06  -2.34* 


.50  -1.00 


.84  2.28 


.89  1.49 
.23  .28 


-  .01    -.  .02 
.38  .35 

-  .35  -1.47 

.03  1.40 

-  .28    -  .72 

j 

-1.42  -2.04* 
17.47 


p<.  05 
'p<  .01 


*** 
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p< 


.001 
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Attitude  Raeulte:     Anelyele  on  Cannon  Stetemente 
Englieh  Coureee.  Alexendrle 
Calendar  Yaar  1976 


1  

Dapandant  Variable: 

Ova  re  11 
Att ltudee 

Couraa 
Satisfaction 

Indlvlduel 
Attention 

Syetev 
Feeturee 

Number  of  Studante : 

440 

4,40 

440 

440 

Multiple  Correlation: 

.48 

.48 

.50 

.41 

b 

t 

b 

t 

b 

t 

b 

t 

Condition 

TICCIT(1)/Lacture(0) 

-6.51 

-4 

50*** 

-3.34 

-5 

.08*** 

-1.37 

-4 .76*** 

- 1 

.28 

-3.54*** 

Entrance  Ability 
Eeeay  Preteet 
Objective  Preteet 

10.  36 
.09 

2 

38* 

99 

5.79 
.05 

2 
1 

.94** 

.28 

-  .09 
.01 

-  .56 
.34 

.23 
.01 

-1.05 
.63 

Dlvialon 

Dey (1) /Evenlng(O) 

5.23 

2 

98** 

2.03 

2 

.55* 

.69 

2.06* 

1 

.37 

3.15** 

Claee 

Laboratory (1) /Regular (0) 

.  34 

21 

.22 

.29 

-  .05 

-  .14 

- 

.54 

-1.31 

Term 

Winter (1) /Fall (-1) /Sprlng(O) 
Spring (1) /Fall (-1) /Winter (0) 

-9.  13 
1.69 

-3 
1 

15** 

92 

-4.7  3 
.83 

-3 
2 

.60*** 
.11* 

-  .11 

.05 

-  .49 
.29 

- 

.30 
.18 

t 

-1.05 
.82 

Instructor 
U 

12 
13 
14 
15 
16 

-  .83 
-8.89  . 
-1.94- 
1.90 
-11.47 
-3.  29 

-3 

-5 
-1 

38 

37*** 

66 

67 

I  7  *** 

37 

.12 
-3.25 

-  .  39 
.  79 

-4.60 

-  .  96 

-2 
-4 

.13 
.  72** 

.  29 
.61 

.58*** 

.88 

-1.28 
-1.48 
1 .07 
.85 
-1.62 
-1 .15 

-3.13** 
-2.98** 

1 .44 

1.57 
-3.88*** 
-2 . 54* 

-2 
-1 

.11 
.09 
.42 
.06 
.94 
.  17 

.20 
-3. 20** 

-  .58 
.08 

-3.55*** 

-  t  29 

Student  Status 

Full-t lme  ClWPart-rlmpfO^ 

2.86 

i 

86 

1.34 

1 

.92  - 

.64 

2.21* 

.45 

1.19 

Enrollment  in  Course 

Repetition  of  course(l)/ 
First  enrollment (0) 

-  .83 

-  . 

36 

-  .07 

.06 

-  .04 

-  .08 

- 

.52 

-  .90 

Employment ' 

Full-time ( 3) /Part-time (2) / 
None(l) 

1.86 

21* 

.73 

1 

.92 

.29 

1.  79 

.58 

2.79** 

Maturity 
Age(years) 

.23 

2 

82** 

.  10 

1 

70** 

.06 

3. 50*** 

.05 

2.20* 

Sex 

Male(l)/Female(0) 

-3.37 

SO** 

-1.27 

-2 

.-40* 

-  .36 

-1.65 

.96 

-3 . 34*** 

OtheV 

Eaaay  Pretest*  Essay  Pretest 
Winter*  Essay  Pretest 
TICCIT*  15 

>.26 

_  i 
3. 

58* 
10** 

-1.23 
1.69 

-3 
3 

21**  ' 
53*** 

-3.06 

-3.27** 

I 

Constant 

\ 

5*. 35 

18.28 

10.  34 

14 

.39 

1 

*p<.05 
** 

p<.01 

*** 

p<.001 


Table  6.  10 
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(-.4662)  23.05      4.50,  where  the  values- within  parentheses  come  from  the 
regression  coefficients  for  the  categorical  variable  of  instructional 

condition  and  for  the  interaction  term.    The?  t-statistic  associated  with 

N 

this  estimated  treatment  effect  is  2.34  (p<.05).    Similarly,  the  estimated 
treatment  effect  on  the  attitude  scale  of  Course  Satisfaction  (CS)  is  « 
(7.9179)  +  (-.2/35)  23*05  =*1.61  with  a,  t-statistic  of  2.31  (p<.05). 
Although  there  is  another  interaction  term  indicating  that  rtale  students 
viewed  specific  features  of  the  TICCIT  system  more  favorably  , than  female 
students  did,  the  estimates  of  treatment  effects  ori  this  attitude  scale  fell 
below  a  statistically  significant  level  for  both  sexes. 

There  were  statistically  significant  differences  in  the  attitudes  of 
students  completing  English  courses  on  the  TICCIT  system'  and  those  of  com- 
parable students  from  lecture-discussion  classes.     Students,  particularly 
younger  students,  reacted  more  favorably  to  classes  on  the  TICCIT  system 
than  to  conventional  sections  of  the  same  English  course;  they  expressed  a 
greater  degree  of  personal  satisfaction  with  the  course  and  believed  a 
greater  amount  of  individual  attention  had  been  given  to  them.    The  esti- 
mates of  treatment  effects  on  the  attitude  score  for  overall  attitudes  and 
the  subscore  for  course  satisfaction,  4.50  and  1.61  respectively,  hold  for 
students  of  the  average  age  in  TICCIT  classes.    These  effects  would  be  even 
larger  for  a  community  college  student  who  was  18  years  of  age:  taq  -  6.85 

and  t      -  2.99.     Since  the  estimated  treatment  effect  varied  inversely  with 
CS 

student  age,  the  size  of  the  effect  diminishes  for  older  students  and 
becomes  negative  beyond  an  age  of  30  years.    Few  students  actually  fell  in 
this  age  group.    As  implemented  in  English  courses  at  Phoenix  College  the 
TICCIT  program  had  a  positive  impact  on  students'  attitudes  toward  their 


course. 
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V.  ' 

When  students  spent  time  on  the  TICCIT  system  as  well  as  attendee! 
regular  classroom  sessions,  the  affective  outcomes  were  quite  different. 
Members  of  the  English  faculty  at  Alexandria  decided  to  use  the  TICCIT 
curriculum  and  still  hold  classroom  meetings.    Although  students  spent  - 
considerable  amounts  of  time  on  the  TICCIT  system  (see  Chapter  7:  Student 
Activities),  these  TICCIT  classes  no  longer  adhered  to  the  principle  of 
mainline  instruction.     Students  learned  both  from  the  TICCIT  program  and 
from  teachers,  without  av^lear  definition  of  the  relationship  between  these 
two  resources •     Individual  instructors  chose  which  courseware  uni tiff*  their 
3t\^d€tit3  would  study:     some  instructors  tried  to  use  the  TICCIT  system  as 
much  as  possible,  and  others  gave  minimal  assignments  in  the  TICCIT  course- 
ware.   As  showfl  in  Table  6.10,  students  reacted  less  favorably  to  an  English 
course  when  it  included  study  on  the  TICCIT  system. 

Other  factors  besides  use  of  the  TICCIT  system  affected  student  atti- 
tudes in  English  courses  at  Alexandria.     Some  of  these  factors  involved 
student  characteristics  while  others  reflected  the  context  of  the  course. 
Older  students  tended  to  react  more  favorably  to  their  course  regardless  of 
its  instructional  condition.     Male  Students  expressed  less  positive  opinions 
about  their  course  than  did  female  students.     There  was  a  curvilinear  rela- 
tionship between  student  attitudes  and  their  scores  on  the  essay  protest: 
students  with  extremely  low  or  high  scores  on  the  essay  pretest  had  lower 
attitude  scores,  on  all  common  statements  (Overall  Attitudes)  and  on  the 
scale  for  Course  Satisfaction,  than  students  with  middle  scores  on  the  essay 
pretest.     Perhaps  both  the  students  least  competent  in  writing  and  those 
already  facile  in  writing  derived  less  benefit  from  their  course  in  composi- 
tion than  they  had  expected. 

) 


Variables  representing  the  context  within  which  a  particular/  section 

> 

was  held  also  contributed  to  the  explanation  of  variance  in  student  atti- 
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tudes.     Students/ in  classes  offered  during  daytime  school  hours  generally 

0   '  '  f 

had  higher  attitude  scores  than  did  students  in  sections  given  in  the 
evening.     Reactions  to  Alexandria's  English  111  were  most  positive  in  the 
spring  quarter  and  least  positive  in  ttytB  winter  quarter,  with  reactions  from 
the  fall  quarter  closer  to  those  of  the  winter  term.     Particular  instructors, 
notably  12  and  15  in  Table  6.10,  also  affected  student  attitudes  toward 
their  course.     Students  with  instructor  15  for  a  section  on  the  TICCIT 
system  reported  receiving  less  individual  attention  than  did  students  with 
the  same  instructor  for  a  lecture-discussion  section  (see  the  regression 
coefficient  in  Table  6.10  for  the  interaction  term,  TICCIT*I5,  on  the 
attitude  scale  for  Individual  Attention). 

The  significance  of  these  student  characteristics  and  context  variables 
in  the  regression  model  does  not  change  the  direction  or  size  of  the  esti- 
mated treatment  effects  reported  in  Table  6.10.     Coefficients  for  the 
dichotomous  variable  of  instructional  condition  still  represent-^pur  best 
estimate  of  treatment  effects.     Other  variables  in  the  regression  model 
simply  contributed  tb  thjv  explanation  of  variance  in  attitude  scores  and  to 
the  identification  of  similar  students  across  conditions.     Besides  the  one 
instructor-treatment  term  applicable  to  a  single  section  with  12  students, 
there  was  no  interaction  which  would  affect  estimates  of  treatment  effects. 

While  student  attitudes  toward  a  course  rose  in  TICCIT  classes  sometimes 
supplemented  by  small  group  discussions,  a  mixture  of  sessions  on  the  TICCIT 
system  and  regular  classroom  meetings  led  to  less  favorable  student  reactions 
In  both  cases  the  standard  for  comparisons  haci  been  student  attitudes  in 
lecture-discussion  sections  of  the  same  English  courses.     That  analyses 
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revealed  statistically  significant  positive  and  "negative  treatment  effects 
seems  neither  inconsistent  nor  contradictory.     The  results  of  the  TICCIT 
program,  like  those  of  any  other  instructional  resource  or  educational  tool, 
depended  on  the  manner  of  its  use. 

*     6.3.c     Estimated  Treatment  Effects 

A  summary  of  the  treatment  effects  estimated  for  student  attitudes 
appears  in  Table  6.11.     Each  estimate  reflects  difference  in  the  attitude 
scores  of  students  completing  a  course  under  the  TICCIT  program  as  contrasted 

with  the  scores  of  similar  students  in  lecture-discussion  sections  of  the 

i 

same  course.     A  positive  treatment  effect  indicates  that  students  in  TICCIT 

1 

classes  had  higher  attitude  scores  than  did  comparable  students  in  lecture- 
discussion  sections,  and  a  negative  treatment  effect  indicates  that  students 
reacted  less  favorably  to  a  course  offered  on  the  TICCIT  system  than  to  the 
same  course  conducted  in  a  traditional  manner.     The  scores  themselves 
stemmed  from  student  responses  to  sta^j^gts  about  the  course,  statements 
phrased  in  an  identical  or  closely  paraiTel  form  across  instructional 
conditions   (see  Tables  6.4  and  6.5).     There  was  one  total  attitude  score, 
called  Overall  Attitudes,  representing  the  extent  of  student  agreement  with 
all  such  statements  as  well  as  three  subscores  representing  students'  reac- 
tions to  specific  aspects  of  a  course_~£s^e  Table  6.1).     Estimates  of  treat- 
ment effects  on  these  scales  reveal  the  extent  of  TICCIT's  impact  on  student 
attitudes  and  identify  the  source  of  that  impact. 

Math  Courses.     Of  the  twelve  analyses  for  mathematics  courses  within 
terms,  there  were  three  significant  treatment  effects  on  students'  overall 
attitudes.     Each  of  these  effects  was  negative:     students  in  TICCIT  classes 
reacted  less  favorably  to  their  math  course  than  did  students  of  comparable 
ability  in  lecture  sections.     That  eight  other  estimates  of  treatment 


/  ,  \  '    -346-  ■; 

/ 

eff  ect£~W^re  also  negative  confirms  the  negative  Impact  of  the  TICCIT 

program  on  student  attitudesTTn  math  courses.     Yet  there  was  /  inly  one  case 

.  *  if 

In  which  students  expressed  significantly  less  personal  satisfaction  with  a 

course  under  the  TICCIT  program.     It  occurred  in  the  first  academic  term  of 
the  demonstration  and  probably  represents  an  overestimate  of  treatment 
effects  given  the  differences  found  In  other  courses  and  subsequent  terms. 
Students'  perceptions  of  a  course  In  relation  to  their  own  n^eds  and  per- 
sonal progress  do  not  seem  to  account  fully  for. the  size  and  direction  of 
the  treatment  effects  on  overall  attitudes.         <\  ^ 

The  reasons  behind  the  less  favorable  reactions  to  math  courses 
under  the  TICCIT  program  become  evident  from  the  estimates  of  treatment 
effects  on  the  attitude  scales  for  Individual  Attention  and.System  Features." 
When  there_was  a  higher  student-teacher  ratio  in  TICCIT  classes  than  in 
regular  classes,  students  in  those  TICCIT  classes  reported  receiving  less 
individual  attention.     Under  such  conditions  the  estimated  treatment  effects 

on  the  attitude  scale  for  Individual  Attention  were  statistically  significant 

* 

and  negative  in  four  cases  out  of  seven.     The  other  three  cases  also  showed 
negative  effects  on  Individual  Attention,  and  these  closely  approached  sta- 
tistically^ significant  levels.    Moreover,  the  significant  effects  occurred 
across  academic  terms  for  mathematics  courses  in  beginning  algebra  (Math  007 
and  Math  108  in  Table  6.11). 

On  the  attitude  scale  for  System  Features  the  estimated  treatment 
effects  were  statistically  significant  and  negative  in  ten  cases  out  of 
twelve  analyses  for  math  courses  within  terms.     The  HELP  sequences  available 
on  the  TICCIT  system  did  not  match  instructors'  explanations  of  difficult 
'  material  in  regular  classes.    Nor  did  the  presentation  of  mathematical  rules 
and  concepts  (RULE)  under  the  TICCIT  program  facilitate  student  learning  to 
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the"  same  extent' as,  classroom  lectures  did  in  traditional  septibns^  These 
comparisons' between  specific  features  of  tjhe  TICCIT^yatenr  and  therir  counter- 
parts4 in  lecture  sections  .depend  on  jparalfcels  drawn/between  certain  aspects 
of^the  .  t;wo  instructional  conditions'    We  belieye  such  parallels  tp  be  war- 


ranted  by  the  a<Jvoca£X  of  the  TICCIT  aptogram  as  /a  complete  and  independent 
replacement  for  conventional  teaching  practices.     Student  react^ojfs  seem  Co-, 
demonstrate  that  there,  is  still  a  need  .f*r  instructor^  explaining  difficult  s 
material  and  presenting  concepts  and  rules.  ,  ^; 

*      Since*  the  estimated  treatment  effects  on  studew  attitude's  in  mafhe- 

.  •    mL  :;$y     ...  •& 

matics  courses  ^apply  to  those  students  who  compld^^^  'course 
under  the  TICCIT  program,  there  was^a  restri^^fairi  .tjfffe Size  of  the  student 
population  that  affected  eadh  arialvsjj^  entrance  ability 


it  affected  ea^h  arialy^^^  entr* 
:ifying  similar!  s tudejai^ d if f  ei 


alone  in  identifying  similar! stnd^^tW^  ln 
their  attitudes  across  conditions •  \Th^  Xtj^et  6i  students  completing  an 


algebra  course  on  the  TICCIT  system  was  SQ  low  at  Alexandria  that  analyses 
of  attitude  data  collected  after  the  ffl^i  quarter  of  the  1975-76  aca4emic 

year  had  to  include^  all  resporidentB^  to  the  student  survey  rather  than  just 

\,  '  \ 

those  vho  had  completed  theii^  cours-e  requirements .    Yet  the  pattern  of 

'*/ 

treatment  effects  femaj^ed  negative  ^nd  still  showed  statistically  signi- 
ficant differences  on  the  attitude  scale  for  system  Features .     In  addition 
to  a  limitation  in  sample  size,,  the  analyses  o^  attitude  results  in  math 
courses  yielded  low  multiple  correlations.     Seldom  did  the  regression  models 
account  for  as  much  as  one-fifth  of  the  total  variance  in  mathematics 
courses.  jf  *  v 

But  these  factors  do  riot  invalidate  t^e ? estimated  treatment  effects. 
They  instead  |make  each  analysis  conservative  ^n  that  it  becomes  harder  to 
demonstrate  a  statistically  significant  . ef  fect.'  .  This  evaluation  ^ias  shown 
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sucrr  eff ects  across  courses  and  academic  terms.     Despite  the  amount  of  ^"V-J^* 
instructor  time  devoted  to  individual  students  (see  Chapter  8:  Faculty 
•Acceptance  and  Tocher  Rol^) ,  students  in  TICCIT  classes  with  a  high  student- 

r 

>         teacher  r^tio  perceived  the  individual  attention  given  them  as/Wss  than 


',f      what  comparable  student-6  in  lecture  sections  reported.     Furtherri&re ,  students 
ratfed  specific  feature$  of  the  TICCIT  system  as  less  helpful  in  their  learn- 

r  ( 

ing  than  the  lecture  Counterparts  of  those  features.     Although  close  in  the 
degree  of  personal  satisfaction  with  a  course,  students  ir!  TICCIT  classes 
,    reacted  less  favorably  to  their  mathematics  course  than  did  students  in 
lecture  sections  of.  the  same  course. 
•  English  Courses'.     Student  attitudes  in  English  courses  on  the  TICCIT 

system  depended  very  much  on  the  context  of  the  particular  course.  The 
estimated  treatment  effects  for  TICCIT  classes  which  sometimes  included 
small  group  discussions  were -statistically  significant"  and  positive  (see 
English  19  and  29  in  Table  6.11).     Those  for  TICCIT  classes  in  which  the  use 
"of  the  TICCIT  system  was  at  the  discretion  of  the  instructor  were  statis- 
tically significant  and  negative  (see  English  111  in  Table  6.11).  Finding 
significant  effects  ilf  opposite  directions  seems  to  demonstrate  that  com- 
puter-assisted instruction  doefl^At  alone  guarantee  any  result.     Instead  its 
impact  depends  heavily  or^  th^manner  in  which  we  choose  to  apply  it. 

On  the  attitude  scale  representing  student  responses  to  all  common 


statements  there  was  a  'del^inite  positive  effect  in  English  courses  at 
Phoenix.     That  positive  treatment  effect  varied  inversely  with  students' 
age:     younger  students,   in  particular,  reacted  favorably  to  English  courses 
when  offered  on  the  TICCIT  system.     Students  in  these  TICCIT  classes 
generally  expressed  stronger  agreement  with  a  statement  about  the  course 
meeting  their  own  needs  than  did  students  in  lecture-discussion  sections 
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(see  response  mean  for  item  6  on  Form  068  in* Table  6.5).  ^Furthermore, 
Students  reported  feeling  a  greater  challenge  to. do  their  best  work  and 
trying  harder  to  learn  the  material  in  TICCIT  classes  (see  response  means 
for  items  19  and  27,  respectively,  on  Form  068  in  Table  6.5).     These  dif- 
ferences stood  behind  the  positive  estimate  of  a  treatment  effect  on  the 
attitude  scale  for  Course  Satisfaction.     The  estimated  treatment  effect 
on  the  attitude  scale  for  Individual  Attention  was  also  statistically 
significant  and  positive.     From  the  student's  perspective  it  seemed  that 
instructors  gave  them  more  individual  attention  and  monitored  their  course 
progress  with  greater  concern  in  TICCIT  classes.     Here  is  the  exception 
which  proves  the  potential  of  the  TICCIT  program,  set  in  an  appropriate 
context,  for  affecting  student  attitudes  in  a  positive  manner. 

Estimated  treatment  effects  for  English  courses  involving  a  combina- 
tion of  regular  classroom  meetings  and  student  sessions  on  the  TICCIT  system 
were  statistically  significant  and  negative.     On  each  attitude  scale  the  dif- 
ference between  scores  from  TICCIT  classes  and  those  from  lecture-discussion 
sections  indicated  more  favorable  student  responses  with  traditional  teach- 
ing practices.     Since  TICCIT  classes  really  entailed  partial  use  of  the 
computer  system,  it  may  seem  that  thejje  comparisons  between  instructional 
conditions  unfairly  take  student  attitudes  toward  a  whole  course  given  in  a 
lecture-discussion  format  as  a  relative  standard  for  evaluating  student 
attitudes  toward  some  fraction  of  a  course  conducted  under  the  TljXIT  pro- 
gram.    But  every  statement  included  on  the  attitude  scales  for  tfourse  Satis- 
faction and.  Individual  Attention  did  deal  with  the  entire  course  and  each 
statement  appeared  with  the  same  wording  in  both  instructional  conditions. 
The  estimated  treatment  ^fects  on  these  two  attitude  scales  were  statis- 
tically significant  and  negative.     When  instructors  decided  to  adopt  the 
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TICCIT  program  as  a  secondary  instructional  resource,  students  expressed 
less  personal  satisfaction  with  the  course  and  reported  a  decrement  in  the 
individual  attention  devoted  to  them. 

boes  the  replication  of  negative  results  in  mathematics  courses  as 
well  as  in  an  English  course  make  the  positive  results  found  in  the  English 
courses  at  one  demonstration  site  less  important?     The  prevalence  of  nega- 
tive estimates  for  treatment  effects,  witt/a  large  number  of  those  effects 
reaching  statistical  significance  on  the  attitude  scales  for  Individual 
Attention  and  System  Features,  may  Wm  to  establish  the  impact  of  the 
TICCIT  program  on  student  attitudes.     Actually  the  negative  effects  instead 
indicated  a  need  to  retain  usual  student-teacher  ratios   in  TICCIT  classes 
and  the  importance  of  the  instructor  in  presenting  concepts  and  in  explain- 
ing difficult  material.     They  perhaps  also  suggested  the  difficulty  which 
instructors  might  encounter  in  attempting  to  incorporate  the  TICCIT  program 
into  their  regular  classes.     The  significant,  positive  effects  remain  impor- 
tant.    They  serve  both  as  an  example  of  the  potential  of  the  TICCIT  program 
and  as  a  reminder  of  the  degree  to  which  context  can  shape  outcomes . 
6.4  Discussion 

The  pattern  of  results  in  the  analysis  of  student  attitudes  clearly 
demonstrates  that  affective  reactions  to  the  TICCIT  program  depend  on  the 
manner  of  its  use.     While  the  estimated  treatment  effects  had  been  consis- 
tently  negative  for  course  completion  rates  and  generally  positive  for 
student  achievement,   those  for  student  attitudes  changed   In  direction  and 
significance  as  the  context   of  TICCIT  classes  became  different.  Moreover, 
the  changes  in  the  estimated  treatment   effects  on  specific  attitude  scales 
fit  the  ron^ltlon^V  courses  offered  on  the  TICCIT  system.     This  aspect 
of  student  performance  h ighl ights   the  importance  of  the  conditions  under 
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which  applications  of  computer-assisted  instruction  take  place,  especially 
the  role  played  by  the  teacher  in  facilitating  student  learning  on  computers. 

Actually,  student  reactione  to  courses  offered  under  the  T1CC1T  program 
were  favorable.     Students  tended  to  agree  with  positive  statements  about 
their  course  (e.g.,  "I  would  recommend  tMs  course  to  my  friends,"  and 
"Instruction  in  this  class  met  my  own  particular  needs")  or  about  the 
TICCIT  program  (e.g.,  "The  rule  statements  RULE  on  TICCiJ  made  mathematical 
concepts  easy  to  learn")  and  to  disagree  with  negativ.    statements  (e.g., 
"The  method  of  instruction  for  this  course  was  too  impersonal  for  me"). 
Furthermore,   the  TICCIT  program  promoted  a  sense  of  student  responsibility 
for  their  own  learning  and  made  students  active  participants  in  the  instruc- 
tional process.     However,  there  vere  differences  in  the  attitudes  of  stu- 
dents completing  courses  under  the  TICCIT  program  and  the  attitudes  of 
similar  students  in  lecture-discussion  sections  of  the  same  courses. 

Of  twelve  analyses  conducted  for  mathematics  courses  within  academic 
terms,   there  were  eleven  negative  estimates  of  treatment  effects  on  overall 
student  attitudes.     The  statistical  test  associated  with  these  negative 
treatment  effects  was  significant  in  three  cases.     Such  results  indicate 
that  students  in  TICCIT  classes  reacted  less  favorably  to  their  math  course 
than  did  students   of   comparable  ability  in  lecture  sections.     Yet  students 
viewed  the  course   itself  and  the  fit   of   Instruction  in  the  course  to  their 

7 

own  needs  nearly  as  favorably   In  TICCIT  classes  as   in  lecture  sections.  On 
the  attitude  scale  representing  students'   personal  satisfaction  witti  a 
course  there  were  still    eleven  negative  estimates  of    treatment   effects  but 
only  one  was  statistically  significant.     Why  students  had  less  positive 
reactions  to  mathemat lcs  courses  under  the  TICCIT  program  becomes  evident 
from  other  attitude  scales. 
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When  the  student-teacher  ratio  in  TICCIT  classes  was  higher  than  that  in 
lecture  sections  of  mathematics  courses,  the  majority  of  estimated  treatment 
effects  on  the  attitude  scale  for  Individual  Attention  were  statistically 
significant ^and  Negative.     Students  reported  that  they  had  received  less 
individual  attention  in  TICCIT  classes,  particularly  in  those  courses  which 
would  be  a  student's  first  mathematics  course  at  the  community  college. 
Instructors  in  TICCIT  classes  had' actually  devoted  a  greater  proportion 
of  their  time  to  counseling  individual  students   (see  Chapter  8:  Faculty 
Acceptance  and  Teacher  Role).     But  computer-assisted  instruction  had  also 
generated  a  greater  student  demand  for  individual  attention.  Faculty 
supervised  students'   final  attempts  at  mastery  tests  and  answered  students' 
questions  about  the  subject  matter.     Whenever  a  studeat  had  to  wait  for  an 
instructor  to  become  available,   that  student  judged  the  individual  attention 
to  be  inadequate.     Although  the  TICCIT  program  indeed  changed  what  teachers 
did  in  mathematics  courses,    it  did  not  lessen  a  student's  need  for  the 

K 

assistance  of  an  Instructor. 

The  estimated  treatment   effects  on  another  attitude  scale,  System 
Features,   suggest  . the  reasons   for  st  intents   frequently  requesting  instructor 
assistance   in  mathematics  courses.     On  this  attitude  scale  there  were  sig- 
nificant negative  effects  evident   across  colleges,   courses,  and  aca'demic 
terms.     Although  students   found   the  practice  problems  on  the  TICCIT  system 
to  be  as  helpful    in   learning  mathematical    rules  and  concepts  as  homework 
exercises,    they   rated   the   rule  statements  and  help  sequences  of    the  TICCIT 
program  less   favorably   than  t  he    lecture  count  erpart  s  \>f    these  system  fea- 
tures.    The  TICCIT  program  had    Incorporated  present  at f ons  of  mathematical 
concepts   at    three    levels  of    difficulty  and  explanations  of    problem  solving 
with  a  fixed  degree  of    detail.     Neither  proved   to  be  the  equivalent  of 
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traditional  teaching  practices.     Classroom  Lectures  and  instructor  explana- 
tions received  higher  student  ratings. 

.<V        English  courses  in  which  each  instructor  decided  on  the  extent  of 
student  exposure  to  the  TICCIT  system  showed  significant,  negative  treatment 
effects  on  every  attitude  scale.     These  TICCIT  classes  really  represented  a 
different  use  of   the  TICCIT  program.     It  was  a  combination  of   regular  class- 
room sessions  and  computer-assisted  instruction.     But  the  English  curriculum 
on  the  TICCIT  system  had  been  developed  as  an  Intact  course.     To  students 
a  mixture  of   learning  on  the  computer  and  teaching   In  a  classroom  may  be 
like  taking  a  single  course  under   two   instructors  with  different  approaches 
to  the  subject   matter.     There  must  be  a  careful  integration  of   the  computer 
and  classroom  curriculum  in  order  to  gain  student  acceptance. 

The  excepfc+Qii  to  these  unfavorable  attitude  results  occurred  in  English 
courses  for  which  the  TICCIT  program  was  the  primary   instructional  resource. 
In  these  courses  on  the  fundamentals  of  grammar  i        the  essentials  of  writ- 
ton  expression,    Instructors  sometimes  adjusted  the  hierarchical  structure  of 
the  TICCIT  curriculum  and  often  conducted  small   group  discussions.  The 
student-teacher  ratio  remained  much  the  same  as   that    In  lecture-discussion 
sections.     Under   these  conditions   there  were  more  positive  student  attitudes 
In  TICCIT  classes    than    in    lecture-discussion   sections.     The  estimated  treat- 
„„.,„    ,.|t«-rt    on  overall    student    attitudes  was  positive  and  statistically 
significant.      Students    in  TICCIT  classes,   particularly  younger  students, 
expressed  a  Kieatei    degree  ol    petsonal.  sat, .tact  Ion  with   their  curse  than 
,,,,,  „!m!l;lr   students    In   rey.ulai    sections.     And  students    in  TTCCIT  classes 
reported  a  K«eater  degiee  ol    Individual   attention  had  been  devoted   to  them. 
'I'd  1 1|  ,o„rse  tonus,    apparently  drew  on  the  strengths  ol    the  TICCIT  program  in 
providing  pu.ctice  cxetcims,    the    hu.t  m.  tors    In  elaborating  on  dltiicul. 
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material  and  iwAvating  students,  and  vgroup  discussions  in  promoting  per- 
sonal interactions.     The  results  were  positive  student  attitudes  and  higher 
posttest  performance  (see  Chapter  5:     Student  Achievement). 

Computer -assisted  instruction  by  itself  neither  guarantees  favorable 
student  attitudes  nor  meets  all  student  needs  for  further  elaboration  and 
clarification  of  the  subject  matter.     Especially  with  a  student  population 
as  heterogeneous  as  that  of  community  colleges  it  would  be  impractical  to 
attempt  developing  a  tutorial  computer  application  capable  of  meeting  the 
needs  of  all  students.     Instructors  must  be  available  for  explaining  what 
students  find  difficult  to  understand.    Moreover,  a  computer  system  that 
supports  the  equivalent  of  full  college  courses  may  relieve  instructors  of 
their  usual  duties,  but  it  also  generates  new  demands  which  preclude  any 
dramatic  increase  in  usual  student-teacher  ratios.    Yet  the  TICCIT  program 
has  shown  that  computer-assisted  instruction  can,  under  the  proper  condi- 
tions, both  facilitate  student  learning  and  prompte  positive  student  atti- 
tudes. 


4  08 


Chapter  7  / 
Student  Activities 

Our  study  of  TICCIT's  impact  on  student  activities  was  intended  to  be 
descriptive  and  somewhat  exploratory.    Unlike  other  aspects  of  student  perfor- 
mance where  the  nature  of  a  positive  effect  may  be  clear,  it  is  often  difficult 
to  identify  and  document  a  desirable  outcome  with  regard  to  activities.  A 
program  might  encourage  students  to  consider  taking  further  courses  or  seek  a  job 
related  to  the  subject  matter,  but  the  real  test  of  such  plans  would  be  in  their 
successful  fruition.     Or  a  curricular  program  would  perhaps  be  conducive  to  more 
frequent  discussions  between  a  student  and  an  instructor  only  to  detract  from  the 
net  time  students  devoted  to  learning  the  subject  matter.     Anji  with  an  innovative 
computer-based  curriculum  such  as  the  TICCIT  program  the  most  pertinent  question 
to  raise  might  simply  be  the  matter  of  what  students  do  on  the  computer  system  so 
as  to  learn. 

This  chapter  has  two  major  sections.     The  first  section  is  similar  to  earlier 
chapters  in  that  it  covers  student  activities  in  a  comparative"  manner .     Data  on 
the  extent  pf  students'  interpersonal  contacts  related  to  the  courses,  the  amount 
of  time  devoted  to  course  studies,  and  their  pi ans^fj&r  further  courses  and  future 
work  will  be  presented  for  students  in  both  TICCIT  classes  and  lecture-discussion 
sections.     However,  there  is  no  attempt  made  to  estimate  treatment  effects  on  such 
activities  since  this  section  is  meant  to  be  descriptive.     The  second  section 
deals  with  the  interaction  of  student  and  computer.     It  concerns  what  happens  to 
a  student  at  a  TICCIT  terminal.     Primarily  that  description  follows  from  student 
use  of  TICCIT's  learner  control  features.     Also  presented  is  some  evidence  on 
the  relationship  between  time  spent  on  the  computer  system  and  learning.  Our 
attempt  to  explore  the  student-computer  interaction  on  the  TICCIT  system  is 
necessarily  descriptive  as  it  relates  only  to  TICCIT  classes. 
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7il    Activities  across  Comparison  Groups 

Part  of  the  student  surveys  (see  Appendix  F)  called  for  responses  to  questions 
about  specific  activities  related  to  a  course  and  possibly  affected  by  the  TICCIT 
program.     The  questions  themselves  reflected  three  central  issues  involved  in 
student  activities.     Firstly,  its  developers  hoped  that  the  TICCIT  program  would 
result  in  a  positive  student  approach  toward  the  subject  matter.     This  would  be 
#hown  by  students  electing  to  take  further  courses  In  the  subject  or  planning  to 
enter  a  related  job.     Actually  data  on  subsequent  course  enrollment  appear 
elsewhere  (see  Chapter  4:     Course  Completion  Rates  and  Appendix  N),  and  here  we 
report  students'  intentions • as  expressed  on  our  survey.     Secondly,  much  criticism 
of  instructional  technology  has  arisen  from  a  concern  about  the  impersonal  nature 
of  media  like  the  computer.     Learning  by  computer,  presumedly,  would  detract  from 
students'  personal  contacts  with  other  students  or  their  instructors.  Again 
pertinent  data  on  students'  own  perceptions  can  be  found  in  another  chapter  (see 
Chapter  6:     Student  Attitudes);  this  chapter  includes  student  estimates  of  the 
frequency  of  their  course-related  interpersonal  contacts.     And,  thirdly,  it  has 
been  said  that  the  amount  of  time  students  devote  to  studying  is  a  critical 
variable  in  their  learning.    Within  this  chapter  we  give  a  basic  description  of 
how  students  allocated  their  study  time  and  later  consider  the  relationship 
between  time  and  learning  for  students  on  the  TICCIT  system. 

Summaries  of  student  responses  to  questions  about  their  course  activities 
appear  in  Appendix  S.     Two  points  should  be  kept  in  mind  while  reviewing  the  data 
in  this  appendix  and  the  tables  which  follow:  only  those  students  who  anticipated 
completing  a  course  responded  to  the  surveys,  except  for  math  courses  at  Alexandria 
where  all  students  attending  classes  at  the  end  of  a  quarter  responded;  and  these 
data  merely  repeat  student  self-reports  in  summary , form.    Thus  the  groups  within 
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each  instructional  condition  may  differ  with  respect  to  a  critical  variable  such 
as  entrance  ability,  on  the  self-reports  may  be  highly  unreliable.    There  would 
be  strong  alternative  explanations  for  any  difference  apparently  due  to  the  mofle 
of  instruction. 

Before  presenting  results  from  a  few  selected  courses  it  might  be  useful  to 
consider  some  irregular  patterns  across  courses.    At  Phoenix,  for  example,  the 
TICCIT  program  apparently  made  the  prospect  of  further  courses  in  math  and  a  job 
related  to  math  seem  attractive  to  students  in  Math  007  during  the  fall  semester 
but  not  the  spring.     Similarly,  a  higher  percentage  of  students  in  TICCIT  classes 
than  lect^upe  sections  of  Math  31  at  Alexandria  indicated  an  intention  to  pursue 
further  math  courses  as  a  result  of  their  course  experiences  in  the  fall  and 
spring  quarters  but  not  in  the  winter.    Most  often  students  in  English  classes 
on  the  TICCIT  system  estimated  more  time  spent  on  their  course  studies  than 
did  students  in  lecture-discussion  classes.     But  the  opposite  result,  students 
spending  less  study  time  in  TICCIT  classes,  prevailed  among  math  courses. 

It  would  seem  that  changes  in  the  direction  of  differences  between  conditions 
require  some  explanation.     In  our  opinion  seeking  underlying  reasons  for  inconsis- 
tent student  self-reports  would  not  be  fruitful.     Such  differences  might  easily 
be  attributed  to  the  unreliability  of  the  estimates  given  by  students  or  a  char- 
acteristic unevenly  distributed  in  the  comparison  groups.     Indeed,  where  changes 
in  the  direction  of  comparisons  do  occur,  they  tend  to  be  small  in  relation  to 
the  n^pber  of  students  involved  or  the  standard  deviation  associated  with  the 
activity. 

Tables  7,1  through  7,4  present  selected  results  for  student  activities. 
These  summaries  correspond  to  the  questions  on  the  student-  survey  explicitly 
concerned  with  plans,   interpersonal  contacts,  and  amount  of  time  devoted  to 
a  course,     A  course  in  each  subject  at  both  colleges  is  represented, 
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Student  Activity  Survey 
Item  Responses  across  Instructional  Conditions 
Phoenix  College 

Hath  007 
Fall  Semester,  1975  < 


IICCIT 
(N  w  27) 


Lecture 
(N  « 134) 


Item  Number 


Item  Number 


Item  Stems  \ 

Plans  for  further  courses  or  work  in  math:  » 

At  the  beginning  of  this  term  did  you  plan  to  take  more  than  just  tile  introductory 

course(s)  in  math?  .  » 

When  you  finish  the  introductory  coutae(s)  do  you  plan  to  take  any  more  advanced  courses  in  math? 

Has  your  experience  in  this  course: 

■  encouraged  you  to  take*  further  courses  in  math? 

•  discouraged  you  about  taking. further  courses  in  math? 

•  had  no  effect  on  your  plans  for  further  courses  in  math? 

Is  a  Job  related  to  math  more  appealing  to  you  now  than  it  was  at  the  beginning  of  this  course? 

Frequency  of  interpersonal  Contact;  (see  explanation  of  codes  given  below): 
During  this  semester  about  how  often  did  you: 

■  meet  with  the 'instructor  [or  proctor]  for  this  course,  outside  of  class,  to  discuss  your 

classvork  or  anything  else  related  to  the  course? 

•  meet  with  the  instructor  for  this  course  to  discuss  personal  or  academic  matters  NOI 

specifically  related  to  the  course? 

•  discuss  questions  related  to  the  course  with  fellow  students? 

•  seek  the  help  of  a  tutor  for  this  course? 

•  use  library  resources  in  connection  with  your  work  in  this  course? 

•  want  to  use  the  TICCIT  system  when  it  wasn't  available? 

Time  spent  on  course: 

Approximately  how  many  hours  per  week  did  you  spend: 

•  working  on  the  TICCIT  system/Attending  classes  for  this  course?  . 

■  in  small  group  discussions  about  this  course  (outside  of  class)? 

•  doing  work  for  this  course  on  your  own  away  from  TICCIT/working  on  homework 

•  in  total  for  this  course?  j 

Item  Coding  (Form  035):     Item  34 (a)-(£) 
3  *  Qtiite  Often 
2  «  Occasionally 
1  *  Almost  Never 


(Form  035) 

N 

Percent 

(Form  040) 

N  Percent 

31 

Yes-17 

63.0 

27 

102  76.7 

No-10 

37.0 

31  23.3 

•30 

Yes-21 

77.8 

26 

111  85.4 

No-  6 

22.2 

19  14.6 

28 

'  13 

48.1 

,56  42.7 

4 

14.8 

10 

37.0 

61  46.6 

33 

v.-  A 

Yes-  j 

JJ.J 

46  3Si4 

No-It) 

84    64.6  . 

Mean 

sd 

Hun  id 

34 

k 

30 

a 

1.22 

.51 

a 

1.26  .52 

b 

1.04 

.19 

b 

1.05  .26 

c 

1.74 

.81 

c 

1.83  .71 

\i 

1.37 

.69, 

d 

1.15  .45 

e 

1.15 

.37 

e 

1.06  .30 

f 

1.93 

.83 

i 

35 

31 

ai 

6.41 

3.02 

a 

2.93  .50 

bi 

.45 

l;io 

b 

.35  .81 

c 

1.10 

2.08 

c 

3.51  2.47 

d 

9.01 

5220 

■  d 

6.66  2.65 

I 

w 

? 
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Table  7.1 
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Student  Activity  Survey 
Item  Responses  across  Imtructlonal  Conditions 
Northern  Virginia  Community  College 
Hath  31 
Fall  Quarter,  1975 


V. 


Item  Stem 

Plana  for  further  couraea  or  work  in  oath: 
At  the  beginning  of  thia  ten  did  you  plan  to  take  more 
than  just  the  Introductory  courae(a)  in  math? 

k 

Vhen  you  finish  the  introductory  courae(a)  do  you  plan  to 
take  any  tore  advanced  couraea  in  uth? 

lea  your  experience  in  thia  course: 

•  encouraged  you  to  take  further  courBea  in  math? 

•  discouraged  you  about  taking  further  courses  in  math? 

'  had  no  effect  on  your  plana  for  further  courses  in  math? 

'Is  a  Job  related  to  math  more  appealing  to  you  nov  than  it 
we  at  the  beginning  of  this  course? 


TICCIT 
(M  a  46) 

Item  Number 
(Fori  063) 


31 
30 
32 

33 


Yes-27 
No-18 


Yes-37 


Percent 

60. 0 
40.0 

60.4 


21 
3 

22 


■  9  19.6 

45.7 
6.5 
47.8 


Yes-18 
No-27 


40.0 
60.0 


Programed 

(N  a  21) ' 

Item  Number 
(Form  164)  N 


31 


30 


32 


33 


^es-14 
No-  7 

Yea-16 
No- 5 

11 
0 

10 


Yes-  6 
No-14 


Percent 

66.7 
33.3 

76.2 
23.8 

52.4 
0.0 
47.6 

30.0 
70.0 


.  Lecture 

(N  *44) 

i 

Item  Number 

(Form  062)   N  Percent 


27 


26 


28 


29 


23 
21 

13 
29 

19 
3 

22 

14 
29 


52.3 
47.7 

31.0  . 
69.0 

43.2 
6.8  * 
50.0 

32.6 
67.4 


Frequency  of  interpersonal  contact:  (see  explanation 
\  of  codes  given  below): 
During  this  qusrtsr  about  how  often  did  you: 
'  meet  vith  the  instructor  [or  proctor]  for  this  course, 
outside  of  class,  to  discuss  your  classvork  or 
anything  else  related  to  the  course? 
1  meet  vith  the  instructor  for  this  course  to  discuss 
personal  or  academic  natters  NOT  specifically 
related  to  the  course? 

•  discuss  questions  related  to  the  course  with  fellow 

students? 

'  seek  the  help  of  s  tutor  for  this  course? 
use  library  resources  in.  connection  with  your  work  in 
this  course? 

want  to  use  the  TICCIT  aystem  when  it  wasn't  available? 
Time  spent  on  course: 

Approximately  how^ny  hours  per  week  did  you  Bpend: 

•  working  on  the  TICCIT  system/attending  classes  for  this 

course? 

in  small  group  discussions  about  this  course  (outside  of 
class)? 

'  doing  work  for  this  course  on  your  own  swsy  from  TICCIT/ 
working  on  homework  assignments? 

•  in  total  for  this  course?. 

Item  Coding  (Form  063):    Item  34(a)-(f) 

3  ■  Quite  Often 
2  ■  Occasionally 
1  »  Almost  Never 


Mean  sd 


Mean  sd 


Mean  id 


9 
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34 


35 
a 

b 

c 
d 


1.45  .66 


09  .29 


1.73  .81 

1.20  .50 

1.13  .40 

1.86  .69 


6.89  2.18 

.25  .73 

1.40  1.68 
6.25  '3.20 


Table  7.2 


34 


a       1.19  .40 


b      4.19  .40 


35 


1.48  .60 
1.05  .22 

1.19  .40 


4.64  .52 

.14  .36 

4.36  4.07 

10.07  5.53 


30 


1.60  .66 


b      1.16  .37 


c  1.93  .84 
I      L10  .30 

e     J.07  .26 


31 

*  4.63  .60 

t>  .60  1.04 

c  ,    5.01  3,36 

<>  10.15  4.91 
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Student  Activity  Survey, 
Item  RetpoMti  acrou  Instructional 

i         Phoenix  Collage 
.    /        English  19  and  29 
Fall  Semester,  1976 


^Iten  Stems 


\ 


Hans  for  further  courses  or  work  in  English:  „ 

At  the  beginning  of  this  term  did  you  plan  to  take  more  than  just  the  introductory 
•  course ($)'  in  English?  I. 

jfan  you  finish  the  introductory  course (s)  do  you  plan  to  take  any  more  advanced 
courses  in  English? 

Has  your  experience  in  this  course: 

•  encouraged  you  to  take  further  courses  in  English? 

•  discouraged  you  about  taking  further  courses  in  English? 

.  had  no  effect  on  your  plans  for  further  cour^  in  English? 


Is  a  Job  related  to  English  more  appealing  to  you 
this  course? 


:iu  in 


(han  it  waB  4t  the  beginning  of 


Condieiont 

v.., 

TICCIT 
(H  *  109) 

Item  Number 
(Form  068) 


Frequency  of  interpersonal  contact:  (see  explanation  of  codes  given  below): 
During  this  semeftter  about  how  often  did  you: 
|  meet  with  the  instructor  [or  proctor]  for  this  course,  outside  of  class,  to 
discuss  your  classwork  or  anything  else  related  to  the  course? 

•  meet  with  the  instructor  for  this  course  to  discuss  personal  or  academic  matters 

NOT  specifically  related  to  the  course? 

•  discuss'questions  related  to  the  course  with  fellow  students? 

•  seek  the  help  of  a  tutor  for  this  course? 

•  use  library  resources  in  connection  with  your  work  in  this  course? 
want  to  use  the  TICCIT  system  when  it  wasn't  available? 

Time  spent  on  course: 

Approximately  how  many  hours  per  week  did  you  spend: 
■  working  on  the  TICCIT  system/attending  classes  for  this  course? 

•  attending  regular  class  meetings,  without  TICCIT,  for  this  course? 

•  in  small  group  discussions  about  this  course  (outside  of  class  periods)? 

•  doing  work  for  this  course  on  your  own  away  from  TICCIT/working  on  horaork  assignments? 

•  in  total  for  this  course? 


£ 

Lecture 
(N*  170) 

Item  Number 
Percent     (Form  067) 


N  '  Percent 


Yei-67 

61.5 

116 

66.3 

34 

NO-A2 

1ft  \ 

Jo.  3 

29 

1  J7 

33.7 

Yes-96 

88,1 

159 

90,9 

33 

No-13 

11.9 

28 

16 

9.1 

• 

56 

51.4 

103 

5B.9* 

35 

1  "- 

.9 

30 

5' 

2.9 

52 

'47.7 

67 

38.3 

Yes-57 

52.3 

oj 

on 

Aft  1 

HO.  J 

36 

No-52 

47.7 

31 

89 

Jiil 

Mean 

so 

Mpfln 
ucaii 

sd 

37 

32 

a 

1.51 

.70 

a 

1.37 

.58 

b 

1,18 

.43 

b 

1.19 

,43 

.  c 

1.99 

..75 

c 

1.85 

,73 

d 

1.45 

.69 

d 

1.12 

.39 

e 

1.19 

.46 

e 

.1.60 

.68 

f 

1.60 

.78 

38 
a 

33 

5.43 

5.05 

a 

2.91 

.71 

b 

1.17 

2.84 

c 

.  .26 

.72 

b 

'  .51 

.99 

d 

,  1,51 

2.58 

c 

3.39 

'  3.00 

e 

9.37 

10.54 

d 

6.77 

5.30 

Item  Coding  (Form  068): 
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Item  37 (a)-(f) 
3  -  Quite  Often 
2  «  Occasionally 
1  «  Almost  Never 
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Ite*  Sum 


Student  Activity  Survey 
Itea  Rasponsei  across  Instructional  Condition! 
Northern  Virginia  Cowunity  College 
Engllih  111 

Pall  Quarter,  1976  T1CCIT  Ucture 

(N*  115)  ,       (N»  74) 

Item  Number  Item  Number 

«  (Form  Q66)      H     Percent     (Form  Q65 )  H 


Plans  for  further  courses  or  work  in  English: 


Percent 


Ac  the  beginning  of  this  tens  did  you  plan  to  take  more  than  Just  the  introductory 

JiHO 

69.6 

45 

60.8 

courie(s)  in  English? 

34 

So-35 

30.4 

29 

29 

39.2 

When  you  finish  the  introductory  courBe(s)  do  you  plan  to  take  any  more  advanced 

Ye»-97 

83.6 

59 

80.8 

course!  in  English? 

33 

No-19 

16}4 

28 

14 

19.2 

His  your  experience  in  this  course!  (see  explanation  of  codes  given  below): 

•  encouraged  you  to  take  further  courses  in  English? 

40 

34.8 

43 

58.1 

Qi9cuuLa(jca  you  aoout  taKin^iurtner  courses  in  tngiisiu 

35 

11 

0  (. 

30 

1 
J 

A  1 

•  'had  no  effect  on  your  plans  for  further  courses  in  English? 

64 

55.7 

26 

37.8 

Ii  a  Job  related  to  English  more  appealing  to  you  now  than  it  was  at  the  beginning  of 

Yea-51 

45.1 

35 

47.3 

this  course? 

36 

No-62 

54.9 

31 

39 

52.7 

Freayency  of  interpersonal  contact: 

Mean 

fid 

Mean 

ad 

DufM  this  semester  about  how  often  did  you: 

37 

32 

•  mt  with  the  instructor  [or  proctor]  for  this  course,  outside  of  class,  to 

discuss  your  classwork  or  anything  else  related  to  the  course? 

a 

1.45 

.62 

a 

1.75 

.68 

•  Beet  with  the  instructor  for  this  course  to  discuss  personal  or  academic  matters 

HOI  specifically  related  to  the  course? 

b 

f  1.25 

.49 

b 

1.17 

.37 

•  discuss  questions  related  to  the  course  with  fellow  students? 

c 

2.14 

.69 

c 

2.11 

.72 

•  seek  the  help  of  a  tutor  for  this  course? 

d 

1.21 

.47 

d 

1.21 

.47 

■  use  library  resources  in  connection  with  your  work  in  this  course? 

e 

1.49 

"  .73 

e 

1.85 

.63 

•  want  to  use  the  TICCIT  system  when  it  wasn't  available? 

f 

1.99 

.80 

Time  spent  on  course: 

Approximately  how  many  hours  per  week  did  you  spend: 

38 

33 
a 

•  working  on  the  TICCIT  system/attending  classes  for  this  course? 

a 

5.19 

3.32 

5.01 

.98 

1  attending  regular  class  meetings,  without  TICCIT,  for  this  course? 

b 

3.05 

1.70 

•  in  small  group  discussions  about  this  course  (outside  of  class  periods)? 

c 

.69 

1.17 

b1 

1.39 ' 

2.57 

'  doing  work  for  this  course  on  your  own  away  from  TICCIT/working  on  homework  assignments? 

d 

3.59 

4.01 

c 

6.59 

5.39 

1  in  total  for  this  course? 

e 

13.74 

9.00 

d 

14.40 

7.80 

u 
ft 

u 


Item  Coding  (Form  066); 


Item  37(a)-(£) 
3  •  Quite  Often 
2  ■  Occasionally 
1  »  Almost  Never 
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Plans  for  Further  Courses  and  Future  Work.     For  a  sizeable  percentage  of 
students  experience  in  a  course  had  no  effect  on  their  plans.     At  least  one-third, 
and  often  one-half  of  the  students  in  a  course,  regardless  of  the  particular 
instructional  condition,  reported  that  their  experience  had  no  effect  on  their 
plans  for  further  courses  in  the  subject.     Seldom  did  a  single  course  discourage 
students  from  pursuing  further  courses:  such  a  negative  effect  on  student  plans 
occurred  infrequently  although  at  times  reached  approximately  an  eighth  of  the 
students  responding.    Of  course  we  should  also  consider  program  outcomes  with 
regard  to  course  completion  in  interpreting  these  data  obtained  primarily  from 
those  students  who  completed  the  requiremeftts  for  course  credit. 

Typically  exposure  to  a  course  on  the  TICCIT  system  encouraged  a  percentage 
of  students  to  take  further  courses  comparable  to  the  percentage  already  encour- 
aged through  lecture-discussion  practice^.     A  similar  conclusion  may  be  drawn 
from  student  responses  to  a  question  about  another  dimension  of  approach  toward 
the  subject  matter.     Course  experiences  made  a  job  related  to  the  subject  matter 
seem  more  attractive  to  roughly  equal  percentages  of  students  in  each  instruc- 
tional condition.     Among  students  completing  a  course,  then,  it  would  appear 
that  the  impact  of  the  TICCIT  program  on  plans  for  further  courses  and  future 
work  was  consistent  with  that  of  control  classes  taught  in  a  conventional 
manner. 

Interpersonal  Contacts.     Perhaps  unexpected  was  the  fact  that  students 
reported  as  much  interperson^  contact  in  TICCIT  classes  as  lecture-discussion 
sections.     In^both  conditions  discussions  of  course  questions  with  fellow  students 
was  the  most  frequent  type  of  contact.     By  contrast,  students  almost  never  sought 
the  help  of  a  tutor  or  met  with  their  instructor  to  discuss  matters  outside  formal 
course  concerns.     Considering  the  obvious  differences  between  a  classroom 
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environment  and  a  computer  system  for  teaching,   there  was  remarkable  similarity 
in  the  frequency  of  students'   interpersonal  contacts.     Admittedly,  our  scale  for 
responses  to  such  questions  was  quite  limited  and  the  questions  about  contact 
with  instructors  dealt  just  with  meetings  outside  of  class.     Still  the  data  tend 
to  counter  arguments  about  the  impersonal  nature  of  computer  applications  in 
education. 

There  was  one  appreciable  difference  in  these  frequency  estimates  that 
seemed  to  follow  from  the  instructional  condition.     But  it  concerned  student  use 
of  resources  rather  than  personal  contacts.     Students  in  English  classes  on  the 
TICCIT  system  reported  less  frequent |use  of  library  resources  in  connection  with 
their  course  work  than  did  students  in  lecture-discussion  classes.     This  may 
reflect  a  qualitative  difference  in  instruction  between  the  two  conditions  or  the 
TICCIT  system's  capacity  to  store  relevant  material  such  as  examples  of  writing. 

It  might  also  be  noted,  with  regard  to  resources,   that  the  majority  of 
students  in  TICCIT  classes  reported  occasionally  wanting  to  use  the  computer 
system  when  it  wasn't  available.     Yet  the  hours  of  system  operation  closely 
matched  the  schedule  of  regular  classes.     Limited  hours  of.  availability  would  not 
seem  to  be  the^reason  behind  these  student  responses.     Instead  it  was  probably 
the  physical  location  that  acted  as  a  constraint  on  availability.     Students  had 
to  be  on  campus   in  order  to  study  on  the  TICCIT  system  while  textbooks  from 
regular  classes  could  be  taken  home. 

Study  Time.     Across  courses  students  in  TICCIT  classes  spent  most  of  their 
study  time  working  on  the  TICCIT  system  while  students  in  lecture  sections 
reported  most  of  their  time  given  to  wording  on  homework  assignments.     So  much 
time  was  devoted  to  homework  assignments  in  lecture  sections  of  math  courses 
that  students  in  TICCIT  classes  spent  less  total  time  on  their  course  studies. 
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Also,  math  courses  courses  on  the  TICCIT  system  adhered  closely  to  the  concept 
of  mainline  instruction:     the  amount  of  timV^fem  working  on  the  TICCIT  system 
accounts  fdr  nearly  the  total  time  spent  on  a  course.     Table  7.1  and  Table  7.2 
represent  the  worst  cases  in  this  regard  since  the  computer  system  was  at  its 
most  incomplete  and  unreliable  stage  of  the  demonstration  period  in  the  fall 
academic  term  of  1975.     Other  cases  contained  in  Appendix  S  better  show  math 
courses'  adherence  to  mainline  instruction.     For  example,  5.93  hours  were  spent 
on  the  TICCIT  system  out  of  the  estimated  6.49  total  hours  per  week  reported  for 
Math  007  in  the  spring  semester,  or  approximately  90%. 

Students  in  TICCIT  sections  of  English  courses  reported  spending  more  time 
than  students  in  lecture-discussion  classes.    An  English  course  on  the  TICCIT 
system  actually  combined  computer-assisted  instruction  with  regular  class 
meetings.    At  Phoenix  students  met  approximately  one  hour  per  week  for  group 
discussions  independent  of  the  TICCIT  system.     At  Alexandria  the  equivalent 
of   three  classroom  sessions  per  week  was  combined  with  extensive  use  of  the 
TICCIT .program.     And  these  English  classes  on  the  TICCIT  system  also  had  writing 
assignments.     Except  for  English  111  in  the  fall  quarter,    the  combination  of 
computer  sessions,  group  meetings  and  homework  assignments  resulted  in  a  heavier 
study  load  for  students  in  TICCIT  classes.     The  estimates  of  study  time  given 
for  English  courses  show  that  this  application  of  the  TICCIT  program  began  to 
depart  from  the  developers'  original  concept  of  mainline  instruction.  However, 
the  greatest  portion  of  study  t.me  in  TICCIT       usses  was  spent  on  the  computer 
system  in  both  math  and  English  courses. 

7.2    Student  Interaction  with  the  TICCIT  System 

Among  the  distinctive  features  of  the  TICCIT  program  were  the  concepts  of 
learner  control  and  mainline  instruction.     Learner  control  gave  students  an 
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opportunity  to  guide  their  own  instruction  and  exercise  control  over  the  learning 
strategies  available  pn  the  TICCIT  system.     Mainline  instruction  meant  that  the 
computer  system  served  as  the  primary  resource  for  teaching.     Much  depended,  then, 
on  what  happened  when  a  student  sat  down  at  a  TICCIT  terminal, .  This  student- 
computer  interaction  would  to  a  great  extent  determine  the  outcomes  of  the  TICCIT 
program. 

Little  was  known  about  what  might  take  place  between  a  student  and  the 
TICCIT  system.     The  instructional  design  of  the  program  was  innovative  and 
experimental.     Yet  the  developers  had  to  commit  themselves  to  that  design  in  the 
hardware,  software,  and , courseware  of  the  TICCIT  system  in  order  to  meet  their 
production  scb'^ule.     The  importance  and  uncertainty  of  student-computer  inter- 
actions prorrpr^.  us  to  take  a  close  look  at  this  process.    We  hoped  that  data  on 
student  use  of  the  TICCIT  system  would  be  helpful  in  understanding  the  program's 

outcomes,  in  refining  subsequent  versions  of  the  system,  and  perhaps  in  designing 

t 

similar  instructional  systems, 
7, 2, a  Observations 

When  the  .demonstration  began  in  the  fall  term  of  1975,  observing  student  use 
of  the  TICCIT  system  was  the  simplest  and  most  direct  method  for  gathering  data. 
Watching  individual  students  as  they  worked  through  TICCIT  materials  enabled  us  to 
look  closely  at  the  process  whose  outcomes  were  under  evaluation.     Observations  let 
us  know  what  the  instructional  design  of  the  TICCIT  system  meant,   in  operational 
terms,  for  the  student.     But  this  procedure  for  collecting  data  on  student-computer 
interactions  also  required  a  substantial  investment  of  staff  resources  in  observing 
students  and  developing  an  appropriate  form  for  recording  data.     It  took  at  least 
half  of  an  hour  to  complete  one  observation.     And  two  semesters  of  pilot  trials 
preceded  the  observations  reported  below, 
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Sampling.     All  students  observed  in  the  fall  academic  term  were  enrolled  for 
the  same  course.     That  course  was  Beginning  Algebra  at  Phoenix  (Math  007)  and 
Algebra  I  at  Alexandria  (Math  31),     Our  observers  randomly  selected  84  students 
at  Phoenix  and  26  at  Alexandria  to  track  throughout  the  term.     It  was  felt  that 
only  a  series  of  observations  on  the  same  student  group  would  permit  us  to  detect 
changes  in  learning  strategies  as  a  function  of  experience.     There  were  five 
observation  cycles  within  the  academic  term.     Each  cycle  lasted  two  or  three 
weeks  and  gave  us  a  minimal  time  period  for  completing  one  observation  on  every 
student  in  the  original  sample. 

Observation  Record.     The  form  used  in  recording  data  on  student-computer 

 ~  \ 

interactions  was  the  product  of  121  trial  observations  and  some  extensive 

revisions.     Its  focus  was  on  obtaining  a  descriptive  record  of  student  sessions 
at  a  TICCIT  terminal.     Questions  requiring  a  subjective  judgment  on  the  part  of 
the  observer  (e.g.,  attentiveness  of  the  student)  were  eliminated.     Coding  became 
straightforward  and  tended  to  follow  an  individual  students  choices  among  the 
TICCIT  options  for  learner  control.     The  directions  given  to  observers  and  the 
form  completed  during  observations  appear  in  Appendix  G. 

The  observation  record  had  an  explicit   coding  scheme  for  describing  student- 
computer  interactions.     Through  codes  specific  to  each  question  the  observers  " 
noted  what  materials  students  brought  with  them  to  the  terminals,  whether  a 
system  failure  occurred  during  the  observation,  how  students  used  learner  control, 
and  what  type  of  assistance  students  sought  beyond  that  provided  by  the  computer 
system.     The  codes  for  these  questions  were  specific  enough  to  insure  a  high 
degree  of  interobserver  agreement.     Indeed  the  average  percentage  of  cases^  in 
which  our  two  observers  assigned  the  sa«  code  exceeded  90%  in  trials  conducted 
to  test  the  form's  reliability  (see  Chapter  2:     Methods  and  Measurement).     As  for 
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the  accuracy  of  these  descriptions,  we  later  note  the  consistency  of  results 
whether  obtained  by  observing  students  at  terminals  or  recording  student  key 
strokes  automatically  by  computer. 

Results.     The  size  of  our  sample  dwindled  with  each  observation  cycle.  Of 
the  84  students  in  the  initial  sample  at  Phoenix  there  was  a  complete  set  of  five 
observations  for  33  students.     The  number  of  observations  fell  steadily  across 
cycles:  there  were  84  observations  in  the  first  cycle,  then  62,   57,   45  and  finally 
40.     Similarly,  only  seven  students  originally  selected  at  Alexandria  had  been 
observed  in  each  of  the  five  cycles.     Part  of  this  reduction  in  sample  size  may 
be  attributed  to  formal  course  withdrawals.     Another  part  was  probably  the  result 
of  our  difficulty  in  locating  students  at  terminals  since  there  was  not  yet  any 
mechanism  for  identifying  students  on  the  system.     But  most  of  the  reduction 
seems  a  function  of   the  self-paced  nature  of  the  TICCIT  program. 

Students  chose  their  own  schedules  for  working  on  the  TICCIT  system.  This 
meant  that  they  often  worked  outside  of  their  assigned  time  periods  or  let  weeks 
pass  without  attending  to  their  studies  under  the  TICCIT  program.     The  tendency 
to  neglect  studies  is  better  evident  in  the  completion  rates  for  math  courses  as 
a  high  percentage  of  reenroll  grades  (see  Chapter  4:     Course  Completion  Rates) 
and  in  the  progress  reports  generated  by  computer  as  the  date  a  student  last 
worked  on  the  TICCIT  system  (see  Figure  7.2).     Irregular  attendance  patterns 
greatly  reduced  the  number  of  observations  done  within  an  academic  term.  The 
completion  rates  for  English  courses  on  the  TICCIT  system,  however,  sug;gest 
that  sample  size  would  have  been  more  stable  had  we  observed  students  in  TICCIT 
English  classes  rather  than  math. 

The  data  from  each  college  had  to  be  aggregated  in  such  a  way  that  a  descrip- 
tive summary  would  be  fair  and  representative.     For  the  Phoenix  observations  it 
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w#s  reasonable  to  aggregate  the  data  according  to  the  five  observation  cycles. 
There  were  so  few  complete  sets  of  observations  for  Alexandria  that  data  had  to 
be  collapsed  into  three  time  periods  in  order    o  act.   n  nearly  20  full  sets.*  This 
left  us  with  only  64  of  the  109  observations     me  at  Alexandria:     20  observations 
were  set  aside  since  they  were  part  of  interobservei.  reliability  trials  and  25 
others  became  duplicates  in  collapsing  data  into  three  cycles. 

Contrary  to  what  would  be  expected  (see  Bunderson,  1975;  Merrill,  1975),  we 
found  no  remarkable  differences  either  across  observation  cycles  or  between  those 
students  with  complete  sets  of  observations  and  others  with  fewer  observations. 
If  learning  strategies  improved  as  students  gained  experience  with  the  TICCIT 
system  or  if  certain  strategies  gave  students  a  higher  probability  for  success 
(i.e.,  course  completion),  such  differences  were  not  evident  from  our  obsegys-  " 
tion  record  despite  its  rather t comprehensive  coverage.     Instead  the  most  obvious 
trait  shared  by  students  completing  a  course  in  introductory  algebra,  aside  from 
pretest  scores  higher  than  their  classmates  (see  Chapter  4:     Course  Completion 
Rates  and  Chapter  5:     Student  Achievement),  was  regular  attendance. 

Table  7.5  is  a  summary  of  student-computer  interactions.     It  follows  the 
major  sections  of  the  observation  record,  beginning  with  "Materials."  Almost 
all  students  at  terminals  brought  some  paper,  whether  in  a  notebook  or  just  loose 
sheets,  for  taking  notes  and  work!  ig  problems.    But  very  few  students  had  the 
course  manual  prepared  by  TTCCIT's  developers.     Faculty  had  recommended  that 
students  purchase  the  course  manual  in  the  summer  semester  but  did  not  do  so 
again  in  the  fall.    Neither  did  students  bring  math  textbooks  to  theirv  sessions 
on  the  TICCIT  system.    They  relied  instead  on  whatever  material  the  computer 
presented  to  them. 
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Studant  Intaraction  with  tha  TICCIT  Syataa: 
Summary  of  Observations 
Alaxandria  and  Phoenix 
Math  31  and  Math  007 
Fall  Term  1975-76 


Phoenix 

Number  of 
Observations 


Total  number  of  observations 
Materials  Present 

!  

Course  manual 

Mote  pad,  notebook 

Work  sheets,  loose  paper 

Math  textbook 

Other  (e.g.,  calculator) 


288 


9 

127 
153 
0 
6 


Percent 
100 


3 
44 
53 
0 
2 


Alexandria 


Number  Qf 
Observations 

64 


1 
43 
18 

0 

2 


Percent 
100 


1 

67 
28 
0 
3 


Total 

Number  of 
Observations 

352 


10 
170 
171 
0 
8 


Percent 

100 


3 
48 
49 
0 
2 


System  Failures 

Frequency 
Duration 


21 

107  min. 


15 

50  min. 


23 


36 

157  min. 


10 


Learner  Control 

Sequence : 

none  observed 
generality  (RULE)  first 
instance  (EXAMPLE,  PRACTICE)  first 
RULE  use: 
frequency — 
0 
1 
2 
3 
4 

pace-<- 

wrote  out  RULE  completely 
took  some  notes  on  RULE 
repeated  the  same  RULE 
read  RULE  at  a  moderate  pace 
merely  glanced  at  RULE 
EXAMPLE  use: 
frequency — 
0 
1-5 
6-10 
11-15 
16- 
pace — 

jfrote  out  EXAMPLE  in  full 

 /''took  sorae  notes  on  EXAMPLE 

repeated  the  same  EXAMPLE 
read  EXAMPLES  at  a  moderate  pace 
merely  glanced  at  EXAMPLES 
PRACTICE  use: 
frequency — 
0 
1-5 
6-10 
Xl-15 
16- 


177 
81 
30 


180 

62 
22 
17 
7 

7 
22 

3 
71 

5 


236 
42 
6 
0 
4 

6 
4 
0 
19 

23 


39 
132 
93 
32 
12 


61 
28 
10 


63 
22 
8 
6 
2 

2 
8 
1 
25 
2 


82 
15 
2 
0 
1 

2 
1 
0 
7 
8 


7 
46 
32 
11 

4 


43 
17 
4 


31 

12 
7 
5 
5 

2 
2 
3 
22 
0 


33 
16 
6 
8 
1 

3 
1 

0 

12 
15 


2 
25 
15 
10 
12 


67 
27 
6 


55 
19 
11 
8 
8 

3 
3 
5 

34 
0 


52 
25 

9 
12 
1 

5 
1 
0 
19 

23 


3 
39 
23 
16 
19 


220 
98 
34 


215 
74 
29 
22 
12 

9 
24 

6 
93 

5 


269 
58 
12 
8 

5 

9 
5 
0 
31 
38 


21 
157 
103 
42 
24 


63 
28 
10 


61 
21 

8 
6 
3 

3 

7 
2 
26 
1 


76 
16 

3 
2 
1 

3 
1 
0 
9 
11 


6 
45 
31 
12 

7 
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Phoenix 

Number  of 

Observations 


Alexandria 


pace — 

■ore  than  5  minutes  per  problem 
3-5  minutes  per  problem 
2-3  minutes  per  problem 
1-2  minutes  per  problem 
less  than  1  minute  per  problem 
HELP  use: 
frequency — 

did  not  use  HELP . 

used  HELP  sometimes  with  incorrect 

responses  to  problems 
used  HELP  almost  always  with  incorrect 

responses  to  problems 
used;  HELP  in  other  ways 
pace —  .     +  ,m 

took  notes  on  HELP 
repeated  the  same  HELP 
compared  solutions  step-by-gtep 
compared  solutions  quickly 
merely  glanced  at  the  answer, 
Use  of  other  TICCIT  features:  j, 
OBJECTIVE 
>  ADVICE 
Introduction 
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Survey 

Requests  for  Assistance 

Frequency  *  ■* 

Questions  ebout  content 
Questions  about  procedures 
Response  by — 
instructor 

proctor  % 

observer, 

•taff 

Type  of  response — 

explanation  or  elaboration 

specific  information 

inadequate  or  respondent  tihable  to 

provide  assistance 
none 


40 
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53 
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18 
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16 
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Percent 
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24 

7 
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90 

26 
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3 

4 

1 

0 

0 
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1 

87 

30 

29 

45 
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33 

51 

18 

13 

20 

64 

18 

36 

12 

16 

25 

52 

15 

20 

7 

14 

22  . 

34 

10 

33 

11 

5 

8 

38 

11 

29 

10 
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9 

35 

10 

7 
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11 

3 

14 

5 

11 

17 

25 

7 

58 

20 

12 

19 

70 

20 

16 

6 

4 

6 

20 

6 

1 

0 

2 

3 

3 

1 

aThis  percentage  assumes  that  each  observation  lasted  20  minutes.     Most  observations  actually  took  longer 
than  that  amount  of  time.'  Several  others,  however,  nnd  to  be  rescheduled  due  to  interruptions  in  system  service 
of  over  15  minutes. 
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System  fai^fr 


es  occurred  fairly  frequently  in  the  f al 1  term  of  the  1975-76 


academic  year.     Their  frequency  actually  interferred  with  collecting  data  at 
Alexandria  in  that  several  observations  had  to  be  discarded  due  to  interruptions 
in  service.     And  at  Phoenix  breakdowns  of  the  computer  system  were  seen  as 


disruptions  in  student  learning  (see  Chapter  6:     Student  Attitudes).     Yet  the 


operating  time.     It  was  the  frequency  rather  than  the  duration  of  system  failures 
that  distracted  students.     In  later  terms,   system  reliability  did  improve,  along 
with  student  perceptions  of  computer  breakdowns  as  a  less  disruptive  influence  on 
their  learning  (see,   for  example,   the  mean  responses  across  quarters  to  item  15 
on  the  student  Survey   for  English  111,   reported  in  Appendix  R) . 

Most  often  there  was  no  sequence  observed  for  students1   use  of  learner 
control.     This  simply  reflects  the  fact  that  observations  most  often  started  when 
students  were  in  the  midst  of  their  work  on  a  segment.     Students  had  to  choose  an 
order  of  presentation  for  the  material  within  each  segment  on  the  TICCIT  system. 
They  might  study  the  general  statement  of  a  mathematical  rule  prior  to  looking 
at  examples  of  its  application  or  trying  practice  problems  requiring  its  use. 
This  deductive  sequence,  RULE  followed  by  EXAMPLE  or  PRACTICE,   was  the  dominant 
approach  in  cases  where  a  sequence  was  apparent.     That  students  should  prefer 
to  study  rules  or  concepts  before  applications  of  those  generalities  stands  to 
reason:   such  a  sequence  is  consistent  with  the  order  of  presentation  already 
familiar  to  students  from  their  textbooks. 

The  low  incidence  of  observed  sequences  meant  that  either  rules  or  instances 
were  absent  from  a  high  percentage  of  observations.  It  was  the  rule.  The  number 
of  observations  in  which  students  failed  to  access  a  rule  statement  (215)  closely 
corresponds  to  the  number  without  a  recorded  sequence  (220).     This  does  not  imply 


\ 


time  lost  to  computer  malfunctions  amounted  to  only  about  two  percent  of  total 


ERIC 


-374- 


that  students  disregarded  generalities.     About  one-fourth  of  the  cases  with 
access  to  rules  involved  writing  out  the  generality  or  taking  notes  on  it  for 
later  reference.     The  lack  of  rule  use,  however,   is  indicative  of  its  importance 
relative  to  the  other  components  of  learner  control.     When  students  resumed  work 
on  a  segment  reft  incomplete  from  their  last  session,  they  seldom  referred  back 
to  a  r?ule  statement. 

Student  use  of  examples  accounted  for  even  fewer  choices  among  TICCIT' s 
options  for  learner  control.     But  here  approaches  differed  by  college.  Fewer 
than  cmef-fifth  of  the  Phoenix  observations  had  some  access  to  example  files  while 
nearl)|  Jne-half  of  Alexandria  data  shows  use  of  examples.     Perhaps  students  at 
Alexandria  were  following  the  sequence  recommended  to  them  by  instructors  as  a 
defaultNprocedure  when  in  doubt  about  a  reasonable  approach  (i.e.,  RULE-EXAMPLE- 
PRACTICE),  while  those  at  Phoenix  employed  a  combination  of  TICCIT  features 
(i.e.,  PRACTICE  together  with  HELP)  that  gave  essentially  the  same  display  as 
an|example.     At  both  colleges  the  dominant  manner  of  students1  use  of  examples 
was  merely  to  glance  at  these  instances  of  rule  applications.     It  would  therefore 
Stfem  that  examples  hardly  represented  the  real  substance  of  the  TICCIT  curriculum. 
Irideed,  their  function  could  easily  be  fulfilled  by  a  combination  of  other  learner 
control  options. 

To  a  student  the  most  useful  component  of  TICCIT1 s  instructional  design 
was  the  practice  problem.     It  was  the  feature  most  often  selected  by  students: 
94%  of  our  observations  indicated  some  use  of  the  PRACTICE  option.  Apparently 
students  chose  to  learn  by  doing.     This  mode  of  learning  also  represented  an 
efficient  use  of  the  TICCIT  system.     In  trying  to  solve  problems,  students 
received  prompt  feedback  on  their  solutions  and,  when  incorrect,  could  request 
a  detailed  explanation  of  the  proper  solution  process.     That  over  half  of  the 
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observations  included  requests  for  additional  explanation,  available  through  the 
feature  called  HELP,   indicates  the  importance  of  these  two  components  of  TICCIT's 
learner  control. 

However,   there  was  a  difference  between  colleges  in  student  use  of  the 
system's  help.     Students  at  Phoenix  referred  to  HELP  quite  frequently  as  an 
explanation  for  solutions  to  practice  problems  and,  maybe,   as  a  replacement  for 
the  example  function  which  illustrates  rule  applications.     Students  at  Alexandria 
tended  to  use  HELP  in  much  the  same  manner,  but  nearly  a  third  of  our  observa- 
tions there  involved  the  use  of  HELP  with  correct  as  well  as  incorrect  solutions 
to  practice  problems.     Even  if  they  had  solved  a  problem  properly,  students 
turned  to  the  system's  solution  as  a  readily  available  check  for  their  own 
answer  and  then  compared  the  two  solutions •     This  would  seem  to  point  to  the 

usefulness  of  feedback  both  in  confirming  correct  responses  and  providing 

/ 

remediation  for  incorrect  responses  (see  Kulhavy,  1977). 

When  the  type  of  assistance  available  on  the  TICCIT  system  waJs  inadequate 
or  inappropriate,   students  sought  help  from  their  instructors  or  proctors. 
There  was  a  feature  of   the  TICCIT  design  intended  to  provide  advice  on  learning 
strategies  and  counsel  students  on  their  course  progress.     But  students  soon 
became  wary  of   the  recommendations  given  as  part  of  ADVICE  after,   for  example, 
taking  a  lesson  test  as  suggested  and  failing  to  attain  a  score  above  TICCIT's 
mastery  criterion.     The  data  from  Phoenix  reflect  this  wariness  as  ADVICE  was 
seen  in  29%  of   the  observations  in  the  first  two  cycles  and  13%  in  the  last  two. 
Students  did,  however,   find  the  summary  of  their  status  helpful,  judging  from 
their  attention  to  this  part  of   the  ADVICE  display  and  the  overall  incidence  of 
ADVICE  in  our  observations.     It   told  them  succinctly  what  TICCIT  features  they 
had  already  seen  (i.e.,  versions  of  generalities  and  instances  at  different 

4-Jj 
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difficulty  levels)  as  well  as  what  options  of  the  learner  control  might  facilitate 
their  learning.     Despite  the  availability  of  HELP  as  a  further  explanation  of  the 
subject  matter  and  ADVICE  for  assistance  in  learning  strategies,   students  asked 
questions   in  about  one-third  of  the  student-computer  interactions . 

Requests  for  assistance,  unlike  other  aspects  of  the  student-computer  inter- 
actions, changed  dramatically  across  observation  cycles.     Initially  fewer  than 
half  of  the  questions  were  about  content.     Most  questions  concerned  procedures 
for  operat  ing^  the  TICCIT  system.     The  responses  to  these  requests  were  typically 
limited  to  the  specific  information  students  needed  in  order  to  proceed.  Toward 
the  end  of  the  terra  there  were  far  fewer  requests  for  assistance  and  most  raised 
points  related  to  the  subject  matter.     Students  had  mastered  the  procedures  for 
learner  control  and  by  then  represented  a  more  select  and  capable  group  in  terras 
of  their  content  knowledge.     Sometimes  a  specific  response  was  sufficient  for 
the  student's  question  but  nea  rly  as  often  it  required  elaboration  of  the  TICCIT 
coverage  by  an  instructor. 

Generally,   these  resuLts  agreed  with  those  obtained  from  pilot  trials  in 
earlier  semesters.     The  one  notable  exception  was  a  lower  frequency  of  RULE  use 
than  had  been  observed  previously,   less  than  38%  in  these  fall  observations  at 
Phoenix  as  opposed   to  over  50%  in  the  preceding  spring  and  summer  terms.  This 
inconsistency  might  be  due  to  the  earLier  necessity   for  students  to  rely  on 
generalities  since  other  components  of  the  courseware  were   incomplete,   or  it 
might   indicate  an  idiosyncrasy  of  the   fall  data.     Neither  interpretation  would 
alter  the  basic  descr ipt  ion  that   emerges   from  these  data. 

The  cornerstone  of   the  TICCIT  system  appears   to  be  the  practice  problems. 
While  the   rules  and   the  examples  oi    their  application  might   easily  be  presented 
in  another  format,    practice  problems  call   the  computer  capabilities   inherent  to 
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ttfl ytlCCt?  system  into  play.     It  was  in  solving  practice  problems  that  students 

-received  feedback  on  their  solutions  and  requested  additional  explanations 
>•••  *        *  ^  \ 

*4  „  available  j£>n  ti*e  system.     It  was  also  with  practice  .problems  that  the  widest 

•jgr^P        spread  cjeourred  in  the  frequency  of  studen£-4^We_jt>f  a  learaer  control  feature. 

And  the  usefulness  of  practice  problems  evident  from  student  choice  of  this 

✓ 

option  fits  well  in  the  context  of  other  outcomes.     It  is  consistent  with  the 
positive  TICCIT  effects  on  student  achievement,  specifically  those  regarding 
the  solution  of  routine  problems  in  mathematics  and,  as  shown  from  our  objective 
£est  of  writing  skills ,  the  identification  and  correction  of  sentence  errors  in 
English  composition  (see  Chapter  5:     Student  Achievement),     Also,  PRACTICE  was 
the  one  system  feature  that  received  high  ratings  by  students  both^.n  comparison 
to  its  closest  counterpart  in  lecture  classes  (i.e.,  homework  assignments)  and 
contrasted  with  ratings  of  other  components  of  TICCIT' s  learner  control  (see 
Chapter  6:     Student  Attitudes). 

It  may  seem  that  practice  problems  represent  an  elementary  application  of 
sophisticated  computer  capabilities.     But  such  applications  work:   their  results 
indicate  that  educational  effectiveness  is  not  a  direct  function  of  technical 
sophistication  (see  Suppes  &  Morningstar,  1969;  Vinsonhaleiy &  Bass,  1972).  The 
enigmas  of  educational  psychology,   such  as  the  diagnosis  a'nd  remediation  of 
student  difficulties  in  learning,   should  represent  enough  of  a  challenge  to  occupy 
us.     The  TICCIT  system  attempted  to  identify  and  anticipate  student  difficulties 
through  its  explicit  content  structure  and  the  isolation  of  critical  attributes 
in  generalities.     Practice  problems  followed  from  these  attributes  and  adhered  to 
the  map  for  content  structure.     Each  problem  had  feedback  on  student  responses, 
and  an  explanation  of   the  correct  solution  process  was  available.     It  would  seem 
that  the  predominant  importance  of   the  practice  problem  may  be  the  reason  for  our 
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f allure  to  record  any  differences  in  students'  learning  strategies,  although 
there  we're  achievement-treatment  interactions  for  course  completion  and  achieve- 
ment as  program  outcomes. 

7.2.-*    On-line  Data 

After  the  fall  academic  term  of  1975  computer  programs  generated  detailed 
reports  on  the  nature  of  student  interactions  with  the  TICCIT  system.  Direct 
observations  then  became  a  redundant  and  expensive  use  of  staff  resources  in  the 
evaluation.     Our  observers  instead  concentrated  on  describing  general  class 
conditions   (see  Chapter  3:     Implementation  of  the  TICCIT  Program)   and  documenting 
instructor  activities  (see  Chapter  8:     Faculty  Acceptance  and  Teacher  Role).  We 
relied  on  reports  produced  on  the  TICCIT  computer  system  to  meet  our  needs  for 
data  on  student  use  of  learner  control  functions.     These  reports  were  counts  of 
the  number  of  times  a  student  referred  to  each  feature  in  TICCIT' s  instructional 
*  design  (see  Figure  1.2).     Additional  on-line  data  were  available  from  progress 
reports  which  gave  a  summary  of  a  student's  course  status  in  terms  of  lessons 
passed  and  time  spent  on  the  TICCIT  system. 

Learner  ControQluncUl^s  :  _  Course  Summaries,     The  most  striking  feature 
of   the  on-line  data 'summaries   Is  the  abundance  ol    numbers  In  them.     But  there 
is  descriptive  evidence   to  he  Klt-anetl   from  the  numbers.     The  particular  course 
summary  lor  student  use  of    learner  control   functions  K^en   In  Figure  7.1 
illustrates  several  points.     This  summary  was   taken  from  system  data  records 
maintained  tor  Kngl  i  f»li   11  1   at   Alexandria  during  the  lull   quarter  of    the  197f.-77 
academic  year.      It   includes  data  on  student-computer   interactions  Kathered  a 
full   year   after   our   observations    In  an  application  of    the  TICCIT  pr.oKram  to  a 
quite  different    content    domain.      Despite   the  abundance  ol    entries,    this  figure 
represents  only  a  partial   summary.     It    covers  six  units  of    the  TICCIT  curriculum 
on  composition  (see  Appendix  H).  '  I  tM 
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Each  entry  in  the  figure  corresponds  to  the  number  of  times  students  chose 

a  learner  CQntrol  function.     For  the  sixth  unit's  seventh  lesscm  (UN  6  LS  7) 

* 

there  were  465  MAP  requests:  the  approximately  300  students  attending  classes 
for  English  111  on  the  TICCIT  system  chose  the  MAP  Qption  for  segments  within 
Unit  6,  Lesson  7  a  total  of  465  times.     The  OBJECTIVE  displays  (OBJ)  for  the 
same  lesson  appeared  212  times,  and  students  sought  TICCIT1 s  ADVICE  (ADV)  291 
times.     Counts  for  RULE  (G),  EXAMPLE  (X),  and  PRACTICE  (Q)  appear  in  sets  of 
five  columns.     These  columns  give  the  total  frequency  of  student  use  (TOT), 
frequencies  for  the  EASY  (EZ  or  E) ,  MEDIUM  (blank  or  M)  ,  and  HARD  (HD)  difficulty 
levels  as  well  as  the  number  of  requests  for  HELP  (H  or  HLP).    ^The  next  two 
column*  (Q+  and  Q-)  represent  the  number  of  practice  problems  students  answered 
correctly  and  incorrectly,  respectively.    The  amount  of  time  that  students  spent 
on  a  lesson  is  given  by  the  last  three  columns.     Unit  totals  appear  as  the  row 
labeled  Lesson  0, 

Although  students  again  chose  practice  problems  more  often  than  any  other 
learner  control  function,  examples  of  rule  applications  also  contributed  to 
student  learning.    Especially  in  those  units  concerned  with  organizing  essays 
and  writing  paragraphs  (Units  9  and  10),  references  to  examples  nearly  equaled 
or  actually  exceeded  those  to  practice  problems.     This  perhaps  suggests  that 
students  chose  to  learn  by  example  when  the  generality  entailed  ambiguity  or 
required  practice  best  done  independently  of  the  TICCIT  computer  system.  But 
the  figures  for  Unit  6  tend  to  disprove  such  an  interpretation  and  illustrate 
the  difficulty  of  attempting  to  draw  inferences  from  these  descriptive  data. 
The  sixth  unit  dealt  with  spelling,  a  topic  in  which  examples  might  be  expected 
to  play  a  significant  role  given  the  rote  nature  of  the  task  and  the  extent  of 
exceptions  to  general  rules.     For  the  lesson  on  plural  nouns  (Unit  6,  Lesson  10), 
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student  use  of  EXAMPLE  was  indeed  greater  than  use  of  PRACTICE,    However,  other 
lessons  (e.g.,  Unit  6,  Lesson  7  on  word  endings  such  as  -tion,  -cian,  and 
-sion)  and  the  unit  totals  (Unit  6,  Lesson  0)  show  a  much  greater  frequency  for 
PRACTICE  than  EXAMPLE. 

Those  units  to  which  students  devoted  the  most  study  time  (Units  8  and  12) 
still  show  an  incidence  of  practice  problems  about  three  times  higher  than  that 
for  examples.     These  units  involved  specific  rules  of  grammer  (for  expanding 
sentences  (Unit  8)  and  analyzing  sentences  (Unit  12).     It  would  seem  that  the 
learning  strategies  adopted  by  students  very  much  depended  on  the  nature  of  the 
content  to  be  learned.     The  instructional  design  of  the  TICCIT  system,  while 
itself  independent  of  subject  matter,  permitted  students  to  pursue  their  own 
preferences  for  learning. 

As  with  our  observations  of  student-computer  interactions  in  introductory 
algebra  courses  earlier  in  the  demonstration,  the  most  popular  choice  among 
TICCIT' s  learner  control  functions  was  PRACTICE.     Again  the  use  of  detailed 
explanations,  available  on  the  TICCIT  system  through  the  HELP  function,  primarily 
accompanied  students'  practice  in  applying  rules.     The  courseware  units  in  which 
students  spent  the  most  time  also  had  the  highest  counts  for  the  PRACTICE  and 
HELP  functions.    And  these  data  from  an  application  of  the  TICCIT  program 
integrated  with  conventional  classroom  sessions  illustrate  the  magnitude  of  the 
instructional  treatment  in  the  evaluation.     For  just  the  seven  units  reported 
for  one  course  in  one  academic  term  (Figure  7.1)   there  were  over  1,500  hours  of 
student  contact  with  the  TICCIT  system. 

Learner  Control  Functions:     Structure*    Data  on  individual  students'  use 
of  learner  control  functions  were  retained  in  the  same  format  as  the  course 
summaries.     From  these  data  we  wanted  to  examine  the  dimensions  underlying 
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TICCIT' s  instructional  design.     The  mass  of  data  made  it  prohibitive  to  study 
auch  structure  in  all  courses.     A  course  had  to  be  selected  that  would  be  a  fair, 
although. perhaps  not  representative,  test  case.    Low  sample  size  excluded  some 
couwes  while  uneven  application  of  the  TICCIT  program  across  classes  disqualified* 
others.     We  chose  to  combine  data  from  Alexandria's  winter  and  spring  quarters 
and  use  on-line  data  from  Algebra  I  in  studying  interrelationships  among  TICCIT' s 
functions  for  learner  control. 

Of  the  courseware  units  required  in  Algebra  I  there  was  evidence  suggesting 
that  the  unit  on  linear  equations  was  particularly  effective.     Student  performance 
on  both  posttests  and  topical  tests  had  shown  an  advantage  for  the  TICCIT  program 
in  material  on  linear  equations  (see  Tables  5.10  and  5.14  in  Chapter  5:  Student 
Achievement).     Therefore,  data  from  that  unit  seemed  to  provide  a  reasonable  basis 
for  exploring  the  structure  of  TICCIT' s  learner  control.     Counts  of  each  student's 
use  of  learner  control  functions  within  Unit  20  led  to  an  intercorrelation  matrix 
which  indicated  the  strength  of  relationship  between  any  two  functus.  This 
matrix  drew  on  data  fr*om  84  students  and  included  18  variables,  essentially  all 
of  those  presented  in  the  on-line  data  summaries  except  for  the  totals  (GTOT , 
XTOT,  and  QTOT)  which  were  redundant. 

A  factor  analysis  according  to  principal  components  resulted  in  a  descrip- 
tion of  the  function  intercorrelation  matrix  in  terms  of  three  factors.  The 
latent  roots  for  these  factors  each  exceeded  an  eigenvalue  of  1.00  (8.22,  2.60, 
and  1.90)  while  no  other  factor  had  a  root  as  high.     Three  factors  were  rotated 
by  a  varimax  orthogonal  procedure.     The  varimax  solution  gave  the  factor  loadings 
shown  in  Table  7.6  and  accounted  for  71%  of  the  total  variance.     Generally  the 
learner  control  functions  had  a  high  loading  under  just  one  factor  and  low  loadings 
under  the  other  two  factors:  the  factor  structure  was  relatively  simple  and  clean. 
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Learner  Control  Functions:    Rotated  Factor  Matrix 
(after  varimax  orthogonal  rotation) 


Unit  20  Linear  Equations 
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As  might  be  expected,  the  factors  themselves  correspond  closely  to  features  of 
TICCIT1  s  instructional  design.  ^ 

The  three  factors  given  in  Table  7.6  lend  substantive  support  to  the 
conceptual  design  of  the  TICCIT  system.     Those  counts  related  to  student  use  of 
the  PRACTICE  function  dominate  the  first  factor.     Thus,  Factor  I  might  be  termed 
a  practice  dimension.     Consistent  with  other  data  on  student   interaction  with  the 
TICCIT  system,   this  factor  has  the  most  descriptive  power.     Most  functions  load 
highly  under  Factor  I,  and  it  alone  accounts  for  40%  of  the  variance  in  student 
use  of  learner  control  options.     Factor  II  clearly  concerns  the  use  of  example. 
It  represents  the  Example  dimension  of  TICCIT1 s  design.     The  highest  loadings 
under  Factor  III,  the  Rule  dimension,  pertain  to  student  use  of  the  RULE  feature. 
However,  the  version  of  the  generality  most  often  seen  by  students,  RULE-MEDIUM, 
appears  to  belong  under  Factor  I,  Practice. 

It  does  seem  that  student  interaction  with  the  TICCIT  system  was  largely  a 
matter  of  work  on  practice  problems.     Under  one  factor  best  described  as  Practice 
come  various  combinations  of  the  practice  feature  (e.g.,  PRACTICE-HARD,  PRACTICE- 
HELP)  as  well  as  functions  related  to  the  structure  and  management  of  instruction 
(i.e.,  MAP  and  ADVICE).     Further,  the  most  frequently  displayed  version  of  the 
rule  statement  and  the  amount  of  time  spent  on  the  computer  system  reach  their 
highest  loadings  under  Practice,  although  both  predictably  show  some  complexity 
in  the  factor  structure  by  their  moderate  loadings  under  other  factors.  Perhaps 
these  split  loadings  serve  as  a  reminder  that  each  dimension,  Practice,  Example, 
and  Rule,  explained  a  substantial  amount  of  variance  in  student  use  of  learner 
control  functions.     And  there  was  certainly  variation  among  students  judging  from 
the  magnitude  of  the  standard  deviations  for  the  functions.     Distributions  of 
counts  for  student  use  of  learner  control  were  skewed  in  the  positive  direction. 
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The  size  of  the  standard  deviations  given  in  Table  7.6  and  the  course 
summary  shown  earlier  in  Figure  7.1  suggest  the  need  to  replicate  these  results 
before  accepting  them  as  definitive  for  the  structure  underlying  the  instruc- 
tional design  of  the  TICCIT  system.     The  standard  deviations  indicate  the  very 
real  possibility  of  unreliable  data.     Some  counts  of  individual  students1   use  of 

learner  control  functions  seemed  extraordinarily  high,  yet  there  was  no  convenient 

/ 

mechanism  for  verifying  them.     Summary  counts  taken  at  the  course  level  gave 
evidence  of  use  of  learner  control  functions  dependent  on  courseware  units  and 
even  lessons.     Factor  analysis  of  other  units  may  yield  different  results.  But 
the  consistency  of  the  structure  found  in  our  analysis  with  the  description  of 
student-computer  interactions  obtained  through  observations  may  instead  point  to 
the  applicability  of  Practice,  Example,  and  Rule  as  dimensions  for  learner 
control  in  math  courses.     At  least  these  dimensions  lead  us  away  from  difficulty 
levels  (i,e,,  EASY,  MEDIUM,  HARD),  system  explanations  (i.e.,  HELP,  ADVICE),  and 
mode  of  presentation  (i.e.,  didactic  RULE  and  EXAMPLE  versus  interactive  PRACTICE) 
as  dominant  facets  of  TICCIT1  s  instructional  design. 

Progress  Reports.     Each  week  instructors  received  a  report  generated  by  the 
TICCIT  computer  system  giving  the  course  status  of  students  enrolled  for  TICCIT* 
classes.     Figure  7.2  gives  an  example  of  such  a  report  from  Beginning  Algebra  at 
Phoenix.     This  particular  report  was  the  last  one  produced  for  the  spring  semester 
of  1976,  and  thus  it  reflects  students1   status  at  the  end  of  an  academic  term. 
It  hardly  presents  a  representative  picture  of  student  progress  since  fewer 
students  completed  their  studies  in  this  class  than  in  other  TICCIT  classes. 
But  it  does  illustrate  the  kind  of  information  regularly  available  to  teachers, 
the  range  of  student  progress  under  a  self-paced  system,  and  the  variation  in 


the  amount  of  time  students  spent  on  the  TICCIT  system. 
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after  receiving  a  non-credit  grade,  in  the  previous  semester. 
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Students  had  to  complete  five  units  (Units  23,   22,  21,  20  and  17)  in  order 
to  receive  credit  for  Beginning  Algebra.     These  units  included  15  lessons  and 
over  80  segments.     So  students  had  to  maintain  a  schedule  equivalent  to  one 
lesson  per  week  or  slightly  better  than  one  segment  per  day  of  classes  throughout 
the  semester.     The  basic  instructions  distributed  to  students  at  the  start  of  an 
academic  term  made  this  point  clearly.     Yet  students  typically  failed  to  pace 
themselves  or  encountered  severe  trouble  in  learning.     For  example,  the  student 
listed  tenth  in  Figure  7.1  had  passed  every  test  on  her  first  attempt  and  had 
done  so  in  a  minimal  amount  of  time.     Certainly  she  was  capable  of  completing 
the  course,  but  stopped  attending  classes  with  over  a  month  still  left  in  the 
semester.     In  contrast  to  this  case,  the  fifteenth  student  attended  classes 
regularly  and  spent  over  60  hours  on  the  system.     And  he  passed  only  four 
lessons.     Even  students  who  had  reenrolled  for  the  course  and  stood  close  to 
completion  (e.g.,  the  third,   fourth  and  fifth  cases  in  Figure  7.2)  neglected 
their  studies  toward  the  end  of  the  term.     Other  students  simply  never  attended 
a  class  or  made  only  a  minimal  effort.     This  lack  of  persistence  among  capable 
students  and  painstaking  progress  among  students  weaker  in  subject  matter 
familiarity  contributed  to  the  low  completion  rates  found  in  math  courses 
under  the  TICCIT  program. 

Part  of  the  explanation  for  student  progress,  such  as  that  depicted  in 
Figure  7.2,  might  rest  with  the  lesson  tests  embedded  in  the  TICCIT  courseware. 
These  tests  were  designed  to  assess  student  mastery  and  a  student  had  to  pass 

each  test  within  three  attempts  in  order  to  proceed  with  his  students.     To  be 

J 

sure  that  students  passed,  all  third  attempts  required  personal  supervision  by 
an  instructor.     Half  of  the  le  ssons  passed  in  our  admittedly  extreme  example 
took  more  than  one  attempt  at  their  respective  mastery  tests,  and  approximately 
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one-third  of  the  successful  attempts  were  third  attempts  guaranteed  to  succeed 
by  the  direct  individual  intervention  of  an  instructor.    This  failure  and 
repetition  of  mastery  tests  was  not  unique  to  math  courses.     It  happened  in 
English  courses  as  well  (as  illustrated  in  the  history  of  test  attempts  for 
Units  8  and  12  in  English  111,  see  Figure  7.1). 

Rather  than  facilitate  student  progress,   the  mastery  tests  on  the  TICCIT 
system  seemed  to  pose  an  obstacle.     A  test  question  taken  from  an  original 
version  of  the  math  courseware  called  for  the  lowest  common  multiple  of  1,568 
and  735.    That  question  occurred  in  the  first  lesson  test  encountered  by  students 
in  introductory  algebra.     It  was  deleted  in  the  courseware  revision  process 
prior  to  the  demonstration,  but  it  still  serves  as  an  illustration  of  a  weak 
link  in  the  curriculum.     The  task  of  test  construction  was  left  to  members  of 
the  production  team  inexperienced  with  such  work,  and  there  was  scant  opportunity 
for  the  developers  to  conduct  a  formative  evaluation  of  test  content  and  mastery 
levels.     Especially  in  the  early  phases  of  the  TICCIT  project,  mastery  testing 
impeded  student  progress.     It  would  be  our  contention  that  further  revision  off 
tests  should  improve  completion  rates.     Of  course,  we  as  evaluators  may  be  prone 
to  see  such  deficiencies  and  be  overly  sensitive  to  them,*  and  thus  find  ourselves 
returning  to  stress  the  difficulty  of  drawing  inferences  from  descriptive 
data  lest  this  comment  be  taken  as  conclusive. 

7.2.c    Time  and  Learning 

The  relationship  between  time  and  learning  (see  Bloom,   1974)  was  a  secondary 
issue  in  this  evaluation.     Like  other  aspects  of  student  interaction  with  the 
TICCIT  system,   there  were  no  grounds  for  comparison  between  student  groups  exposed 
to  the  TICCIT  program  and  others  taught  by  conventional  methods.     Yet  the  nature 
of  the  TICCIT  program  gave  us  an  unusual  opportunity  to  collect  data  on  the 
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amount  of  time  students  spent  in  learning.    Measures  of  that  learning  were 
already  a  part  of  the  evaluation.    And  the  resulting  evidence,  however  imperfect 
and  tentative,  warrants  presentation  here.     Contrary  to  popular  expectations  we 
found  no  strong  relationship  between  time  spent  in  learning  and  external 
measures  of  student  achievement. 

Time  Spent  on  the  TICCIT  System.     Course  averages  for  the  amount  of  time 
that  students  spent  on  the  TICCIT  system  appear  in  Table  7.7.     Calculated  for 
all  students  who  attended  classes,   the  highest  mean  occurred  in  English  111. 
Students  in  a  course  which  combined  the  TICCIT  program  with  traditional  class- 
room practices  spent  more  time  on  the  computer  system  than  students  in  courses 
representing  mainline  applications  of  the  TICCIT  program.     It  is  true  that  this 
course  met  five  times  per  week  while  other  courses  carried  three  credits.  But 
the  shorter  academic  term  of  a  quarter  probably  compensates  for  this  difference. 
The  real  reason  for  the  higher  means  in  both  English  English  19  during 

the  fall  academic  term  might  be  the  extent  of  instructor  supervision.     By  then 
instructors  were  familiar  with  t ^e  TICCIT  English  curriculum,  and  these  courses 
particularly  represented  a  mixc  re  of  classroom  meetings  or  small  group  discus- 
sions with  sessions  on  the  TICCIT  system.     With  regular  instructor  contacts 
outside. of  TICCIT  sessions  students  perhaps  felt  responsible  for  their  own 
learning  and  answerable  to  a  specific  faculty  member. 

There  was  substantial  variation  among  students  in  the  amount  of  time  spent 
on  the  TICCIT  system.     Some  students  spent  virtually  no  time  working  on  the  TICCIT 
system  while  o t he ^ devoted  a  considerable  number  of  hours  (see,  for  example, 
Figure  7.2).     Those  students  who  took  posttests  did,  on  the  average,  spend  more 
time  than  all  students  enrolled  in  a  class  regardless  of  their  attendance  or 
course  progress.     Indeed  the  means  for  students  with  posttests  indicate  attendance 
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TIM  Spent  on  the  TICCIT  Syste«:    Student  Contact  Houri 
Descriptive  Statlatlca  for  Selected  Couress 
.  Calendar  Year  1976 


CourM 


Mtth  007 
Hath  31 
Math  31 
lotllah  19 
enfliah  29 
lafllah  19 
ftagliah  29 
English  111 


Term 

All  Students 
N      Mean  sd 

Student*  with  Posttest 
N      Mean  erf 

Preteat 

Correlation* 
Poatteat 

Spring  Semeeter . 

159 

21.02 

20.7 

39 

38.64 

26.4 

.00 

-.14 

Winter  Quarter 

98 

26.88 

22.5 

54 

35.93 

25.1 

-.02 

.30 

Spring  Quarter 

53 

23.94 

17.4 

27 

34.15 

17.5 

.06 

.06  - 

Spring  Semester 

72 

20.42 

16.3 

20 

30.55 

19.1 

.22(-.05)* 

-.18(.14)* 

Spring  Semester 

60 

22.08 

21.0 

(*28 

35.14 

21.7 

.15(.21) 

.24(-.10) 

Fell  Semester 

166 

29.64 

16.1 

82 

38.94 

11.8 

~.08(-.09) 

-,29(-.30) 

Fell  Semester 

108 

24.16 

15.4" 

40 

33.67 

10.8 

-.23(-.21) 

-,40(-.34) 

Fell  Quarter 

105 

30.13 

23.3 

66 

37.14 

22.8 

-.09(-.14) 

-.33(-.15) 

*Th*  correction,  within  parentheses  refer  to  the  relationship  between  time  and  essay  tests 
while  those  without  parentheaea  refer  to  ti«e  and  objective  tests. 
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.60 
.75 
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equivalent  to  three  and  sometimes  over  four  class  periods  per  week.     That  the 
standard  deviations  for  these  cases  remain  large  might  be  taken  as  a  sign  that 
the  time  spent  was  related  to  entrance  ability  or  contributed  to  posttest 
performance.     Either  Interpretation  would  be  Incorrect. 

Time  and  Test  Performance.     It  might  be  suggested  that  students  weaker  In 
terms  of  their  entrance  ability  would  need  to  spend  more  time        the  TICCIT  y 
system  In  order  to  attain  mastery.     Thus  there^should  be  a  negative  correlation 
between  time  on-line  and  pretest  scores.     Or  the  position  adopted  might  be  that 
students  of  comparable  entrance  ability  would  differ  In  their  posttest  perform- 
ance as  a  direct  function  of  the  amount  of  time  they  had  spent  on  the  TICCIT 
system.     There  should  then  be  a  positive  correlation  between  time  on-line  and 
posttest  scored  or  a  positive  partial  correlation  between  these  two  variables 
adjusted  for  pretest  scores.     The  correlations  given  in  Table  7.7  support  neither 
position.     And  regressions  similar  to  those  reported  for  student  achievement  (see 
Tables  5.5,  5.6,  5.12  and  5.13  in  Chapter  5:     Student  Achievement)  but  conducted 
just  for  students  within  the  TICCIT  condition,   failed  to  show  one  instance  in 
which  time  on-line  contributed  to  the  explanation  of  variance  in  posttest 
performance . 

Sine* : the  potion  that  time  and  learning  should  be  related  in  a  mastery 
system  has  such  great  intuitive  appeal,   finding  no  appreciable  relationships 
between  time  on-line  and  test  performance  came  as  a  surprise  to  us.     But  the 
finding  is  less  puzzling  in  the  context  of  other  results.     There  was  evidence 
of  an  achievement-treatment  interaction  both  in  the  selective  nature  of  the 
TICCIT  program  with  regard  to  completion  rates  in  math  courses  and  in  the  role 
played  by  interaction  terras  in  the  analyses  for  completion  and  achievement  in 
English  courses  (see  Chapter  A:     Course  Completion  Rates  and  Chapter  5:  Student 
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Achievement).  This  perhaps  suggests  that  the  TICCIT  program  provided  insufficient 
instructional  support  for  students  weak  in  entrance  ability. 

We  should^expect  to  find  a  relationship  between  time  and  learning  only  under 
conditions  in  which  the  instructional  treatment  changes  as  a  function  of  time. 
Simply  proceeding  at  a  slower  pace  repeating  the  same  instructional  sequence  would 
not  be  likely  to  benefit  students  incapable  of  understanding  the  n>  '  erial  upon 
first  exposure.     The  nature  of  the  treatment  must  also  change.    Ye:,  the  very  stu- 
dents who  might  otherwise  derive  the  most  benefit  from  additional  time  on  the 
TICCIT  system  probably  lacked  the  sophistication  to  manipulate  the  system  through 
learner  control  so  as  to  meet  their  own  needs,  assuming  that  such  manipulation 
was  possible.     Indeed,  aggregating  our  observations  of  students  at  TICCIT  termi- 
nals either  by  cycles  within  a  term, or  by  matches  for  a  student  across  cycles  had 
\ot  shown  any  consistent  pattern  of  differences  in  students'   learning  strategies. 
Yet  the  data  on  time  and  learning  were  equivocal.     Taking  a  measure  of 
student  learning  from  system  records  we  found  a  consistent  positive  relationship 
between  time  on-line  and  lessons  passed  (see  Table  7.7).     The  amount  of  time 
students  spent  on  the  TICCIT  system  contributed  significantly  to  the  explanation 
of  variance  in  the  number  of  TICCIT  lessons  students  had  passed.    This  held  for 
all  the  courses  listed  in  Table  7.7  in  analyses  which.,  also  included  pr^st  as 
an  explanatory  variable.     It  would  be  tempting  to  diflount  suCh  resultf||s 
inherent  to  the  use  of  an  achievement  measure  taken  from  the  TICCIT  system. 
But  the  analyses  with  this  internal  measure  of  achievement  included  substantially 
larger  data  sets  than  those  with  our  external  measure  of  achievement,  the 
posttest.     For  example,  there  were  ft  students  in  Math  007  for  the  spring 

„ester  with  data  on  lessons  passed,  time  c/n-line  and  pretest  but  only  32  wit 
posttest,,  time  on-line  and  pretest.,  Whether  there  was  a  relationship  between 
time  and  learning  depends  on  which  measure  of  achievement  we  choose  to  accept. 
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Chapter  8 
Faculty  Acceptance  and  Teacher  Role 

Thus  far,  this  report  has  focused  on  evaluating  the  impact  of  the 
TICCIT  program  in  terms  of  its  effects  on  student  performance.     The  present 
chapter  focuses  on  the  program  in  relation  to  the  teacher.  Specifically, 
this  chapter  addresses  the  issues  of  the  teacher's  role  under  the  TICCIT 
program  and  faculty  acceptance  of  the  program.    Describing  what  teachers 
did  in  classes  offered  on  the  TICCIT  system  is  pertinent  for  both  understand 
ing  the  context  of  the  program's  impact  on  student  performance  and  appreciat 
ing  the  nature  of  the  program's  demands  upon  teachers.     Docum^^ig  faculty 
beliefs  about  educational  practices  and  computer-assisted  instruction  and 
later  their  reactions  to  the  TICCIT  program  gives  us  the  benefit  of  the 
colleges'  initial  views  on  the  project  and  its  premises,  and  of  the  colleges 
subsequent  opinions  based  on  experience  with  the  program. 

Two  major  sections  appear  in  this  chapter.     The  first  section  presents 
questionnaire  and  observation  data  relevant  to  faculty  activities.  Instruc- 
tors of  target  courses  in  English  and  mathematics  estimated  what  percentage 
of  their  time  had  been  devoted  to  various  tasks,  such  as  preparing  for 

class  and  counseling  students  on  questions  about  course  content.     Also,  the 

t 

tasks  performed  by  instructors  in  TICCIT  classes  were  recorded  through 
direct  observations.     The  second  section  describes  faculty  attitudes 
prior^a^the  installation  of  TICCIT  systems  at  the  colleges  and  documents 
their  ^^yt  ions  to  the  program  after  nearly  two  years  of  experience  v^^T^v! 
it.     These  descriptions  of  faculty  opinions  and  judgments  come  fromrithe  ' 
results  of  surveys  conducted  at  two  different  points  in  the  project's^ 
history.     This  chapter,  like  the  previous  one  on  stii&ent  activities,  was 
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meant  to  be  descriptive  and  includes  no  formal  estimates  of  treatment 
effects.     Instead  it  provides  a  rather  comprehensive  picture  of  the  TICCIT 
program  in  terms  of  faculty  duties  and  attitudes. 

8. 1    Teacher  Role 

There  has  been  considerable  speculation  about  what  the  role  of 
the  teacher  would  be  when  technology  became  commonplace  in  our  schools. 
Often  it  is  said  that  technology  will  relieve  teachers  of  routine  duties 
and  thus  make  better  and  fuller  use  of  their  capabilities.     This  interpre- 
tation of  technology's  impact  on  teachers  might  be  restated  in  a  simple, 
direct  manner:     "They  will  be  freed  to  do  the  things  they  have  long  wanted 
to  do  but  never  had  the  time"  (Gillett,   1973,  pp.  82-83).     Other  projections 
involve  radical  changes  in  teachers'  responsibilities.     For  example,  one 
report  forecasted  a  shift  in  the  teacher's  role  from  surveyor  of  information 
to  master  of  the  resources  of  learning  (Kerr,  1972).     The  view  of  technology 
held  by  Norris  (1977)  incorporates  the  themes  of  both  relief  and  change. 
He  foresaw  technology,  more  specifically  computer-based  education,  givfng 
teachers  more  time"  to  work  with  individual  students  and  creating  demands 
for  new  curricular  materials  produced  by  teachers. 

Its  developers  had  anticipated  that  the  TICCIT  program  would  change 
the  nature  of  a  teacher's  duties  and  responsibilities..   They  had  addressed 
this  issue  in  the  early  stages  of  project  planning  (e.g.,  Black,  1974; 
Bunderson,   1973).     But  it  was  quite  difficult  to  define  the  teacher's  role 
without  the  benefit  of  some  experience  in  teaching  by  mainline  computer- 
assisted  instruction.     The  developers  instead  offered  a  broad  portrayal  of 
possible  duties,  as  shown  in  Figure  8.1.    Actually  these  projected  duties 
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*Taken  from  TICCIT  Community  College  Implementation  Flan  (2nd  ed.),  1974,  Appendix  I,  page  34. 
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Figure  8.1* 
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methods.     Elaborating  on  t\ie  content  pf  TICCIT' courseware,,  for  example, 

sight  *e .  likened  to^^  of  a  topic,  ( 

£%98i8$3K  |J^ent^  qffr  a  "third  attempt  ,tef\a  mastery  test  would  perhaps 

parallel  courtsfeling  students  on* 4sbeciall)r>oor  test  performance.  Despite 
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such  similiarities  there  should  be  marked  differences  inf.  the  relative 

demandj^-^f  teacher  t4sks  if  the  TICCIT  program  succeeded  in  supplanting 

some  rout ine^JLns true tional  responsibilities •  -^^j 

The  approach  followed  here  in  documenting  the  teacher's  role  involves 

comparisons  across  conditions  as  well  as  records  of  activities  from  just 

TICCIT  classes.     We  can  begin  to  define  what  a  teacher  does  under  the 

TICCIT  program  by  noting  points  of  similarity  and  contrast  with  those 

activities  normally  associated  with  teaching.     A  questionnaire  on  faculty 

.activities  (see  Appendix  J)  provided  a  data  base  sufficient  for  this 

purpose.     Still,  faculty  estimates  of  the  proportion  of  their  time  spent  in 

various  activities  may  be  just  rough  figures  and  convey  only  a  sense  of 

teacher  role.     A  more  complete  description  of  teacher  activities  can  come 

from  direct  observations.     Within  TICCIT  classes,  we  observed  the  tasks 

performed  by  instructors  while  students  worked  at  the  computer  terminals. 

The  description  of  teacher's  role  that  emerged  from  these  observations 

supplements  the  results  of  the  self-report  questionnaire  in  that  it  gives 

an  independent,  objective  account  of  what  teachers  did  under  the  TICCIT 

program. 
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8.1. a    Instructor  Activities  across  Conditions 

The  activity  questionnaire  made  it  possible  for  us  to  contrast  the 
duties  fulfilled  by  teachers  conducting  TICCIT  classes  with  those  associated 
with  teaching  the  same  course  in  a  conventional  manner.     Faculty  responses 
to  the  questionnaire  simply  reflected  their  own  estimates  of  how  they  had 
spent  their  time.     Decisions  regarding  what  pattern  of  time  allocations 
might  best  be  suited  to  mainline  computer-assisted  instruction  were  usually 
made  by  an  individual  faculty  member  in  accordance  with  the  demands  of  the 
situation.     And  the  situation  was,  to  a  large  extent,  a  function  of  depart- 
mental policy  on  student-teacher  ratio,  course  requirements,  and  assigned 
classroom/ computer  hours.     The  same  combination  of  individual  autonomy  and 
departmental  policy  can  be  said  to  affect  instructor  activities  in  courses 
taught  by  familiar  lecture-discussion  practices.     Just  a     it  wo^ld  be 
speculative  to  attribute  any  of  the  effects  of  the  TICCIT  program  to  a 
particular  component  of  the  system,  it  would  likewise  be  inappropriate  to 
consider  any  change  in  instructor  activities  as  an  effect  of  the  TICCIT 
program  rather  than  as  a  consequence  of  faculty  decisions  or  departmental 
policy. 

These  data  on  teacher  role  represent  what  happened  at  the  community 
colleges  involved  in  the  TICCIT  demonstration.     They  do  not  necessarily 
represent  the  role  model  .advocated  by  the  developers  or  the  procedures  that 
other  colleges  should  try  to  follow.     Instead,  the  data  from  the  instructor 
activity  questionnaire  should  help  in  understanding  the  context  within 
which  the  TICCIT  program  had  significant  impact  on  student  performance. 
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Activity  Questionnaire.    The  questionnaire  designed  for  collecting 
data  on  instructor  activities  had  essentially  two  .parts,  one  specific  to 
course  sections  and  another  on  general  types  of  activities  perhaps  unrelated 
to  a  particular  course  (see  Appendix  J).    The  first  part  specific  to  course 
sections  dealt  with  instructors;  use  of  classroom  hours  and  the  proportion 
of  their  time  given  to  various  tasks.     The  second,  general  part  concerned 
the  amount  of  time  faculty  spent  on  college  committees,  student  organizations, 
further  education  in  their  subject  field,  informal  discussions  with  students 
and  colleagues,  and  similar  duties  involved  in  teaching.     This  second  part 
was  included  in  the  questionnaire  in  order  to  check  on  instructors'  use  of 
somewhat  discretionary  time,  since  the  TICCIT  program  might  'free'  instruc- 
tors to  engage  in  activities  aside  from  those  tied  to  specific  course 
sections . 

Sample.     Faculty  members  who  taught  sections  of  target  courses 
received  the  instructor  activity  questionnaire  at  the  end  of  the  academic 
terms  of  the  1975-76  school  year.     This  was  very  much  a  fixed  sample  of 
respondents:     there  were  less  than  10  faculty  members  in  each  college's 
math  department  and  fewer  than  20  instructors  itt  each  English  department. 
And  not  all  of  these  faculty  members  taught  sections  of  courses  designated 
for  evaluation.     The  sample  of  English  instructors  was  further  limited  by 
the  fact  that  the  demonstration  period  for  TICCIT  courseware  for  English 
began  in  the  second  half  of  the  1975-76  school  year.     Still,  instructors 
completed  and  returned  40  questionnaires.     Of  these,  28  forms  were  from  math 
instructors  and  12  from  English  instructors;  one-fourth  of  the  respondents 
were  part-time  instructors.     It  might  also  be  noted  that  these  questionnaires 
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were  collected  after  the  developers  of  the  TICCIT  program  had  given  orienta- 
tion sessions  to  train  faculty  in  the  use  of  the  TICCIT  system.  Still, 
faculty  responses  often  represented  what  had  happened  in  their  first 
practical  exposure  to  teaching  with  the  TICCIT  system. 

Questionnaire  Data:     Activities  specific  to  Course  Sections. 
A  summary  of  the  time  allocations  for  tasks  specific  to  courle  sections 
appears  in  Table  8.1.     This  summary  covers  instructor  activities  in  math 
courses  by  college  sin  :e  instructors  at  Phoenix  taught  an  assigned  number 
of  hours  on  the  TICCIT  system  and  instructors  at  Alexandria  taught  specific 
TICCIT  sections  of  math  courses.     The  descriptive  statistics  on  instructor 
activities  in  English  courses,  however,  appear  for  both  colleges  taken 
together.     Twelve  responses  from  one  academic  term  of  the  demonstration 
period  for  English  courses  did  not  warrant  a  delineation  by  college.  The 
modes  of  instruction  and  the  types  of  activities  listed  in  the  table 
correspond  to  those  given  in  the  questionnaire. 

Clearly  the  treatment  was  implemented.     Math  instructors  spent  over 
t^ve  classroom  hours  per  week  with  the  TICCIT  system,  and  English  instructors 

had  the  equivalent  of   three  class  sessions  per  week  on  the  TICCIT  system. 

\  ^  ^  - 

By  contrast,  ins  true  to he  devoted  therlr  classroom  hours  to  lecture  and 

^ibns   taught   in  a  conventional  manner.     From  the 

total  number  of  t las<room\hours  it  can  be  seen  that  the  math  department  at 

\ 

/   Phoenix  considered  two  sessions  on  the  TICCIT  system  to  be  the  same  as  one 
regular  classroom  session,  policy  was  adopted  in  order  to  compensate 

for  the  homework  assignments  done  for  lecture-discussion  sections.  At 
Alexandria  the  total  number  of  classroom  hours  for  math  courses  indicates 
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Instructor  Activitieg  Survey 
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(a)  Planning  for  class 

0 
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2.74 

6.21 
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1.57 

4.16 

7,00 

8.37 

3.57 

7.48 

(b)  Conducting  class  , 

16.25 

21.36 

51.67 

20.72 

27.93 

33.69 
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6.26 
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29.83 

26.71 

16.43 

(c)  Preparing  lectures,  discussions 

0 

0 
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(d)  Developing  student  assignments, 
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/     on\Weir  course  progress  \ 
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12.00 
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(g)  Counseling  students  on 

content  questions 

34,50 
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40.93 

31.27 

6.71 

3.45 
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(h)  Counseling  students  on  matters 
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that  the  math  courses  there  were  five-credit  courses  which  met  five  times 
per  week.     These  figures  on  the  use  of  classroom  hours  also  make  it  apparent 
that  instructors  of  TICCIT  sections  of  English  courses  supplemented  the 
computer-based  curriculum  with  their  own  discussions  and  sometimes  a 
lecture . 

The  data  on  the  percentages  of  an  instructor's  time  spent  on  various 
tasjcs  (Table  8.1)  reveal  a  quite  different  pattern  of  activities  across 
conditions.     In  math  courses  instructors  with  lecture  sections  usually 
spent  two-thirds  of  their  time  in  conducting  class,  preparing  lectures,  and 
developing  student  assignments  and  tests.     The  TICCIT  program  cut  this 
preparatory  and  didactic  work  down  to  one-fourth  or  less  of  an  instructor's 
role.     Instructors  with  TICCIT  classes  instead  devoted  much  of  their  time 
to  counseling  students  on  content  questions.     At  least  in  this  sense  the 
TICCIT  program  did  free  instructors  to  work  with  individual  students. 

That  instructors  of  math  courses  on  the  TICCIT  system  spent  a  consider- 
able  portion  of   their  time  on  individual  s\udenta'y  content  questions 
perhaps  supports  other  findings.     It^ay  signify  the  difficulty  encountered 
by  students   in  understanding  matheraat teal  concepts  as  presented  in  the 
TICCIT  program  and,  therefore,  the  lower  course  completion  rates  in  TICCIT 
math  courses.     It  may  also   inriLcate  the  assistance  which  enabled  othe* 
students  not  only  to  complete  a  math  course  on  the  TICCIT  system  but  to 
gain  higher  scores  on  achievement  posttests  as  well. 

The  pattern  of  activities  reported  by  English   Instructors  with 
TICCIT  classes  seems  to   resemble  the  estimates  given  by  math  instructors 
with  lecture  sections   (see  Table  8.1).     Nearly  one-half  of   their   time  was 
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devoted  to  conducting  classes  and  one-tenth  to  correcting  student  assignments 
English  instructors  with  lecture-discussion  classes,  by  contrast,  spent 
just  one-fourth  of  their  time  conducting  classes,  but  developing  classroom 
presentations  and  student  assignments  took  another  fourth  of  their  time. 
Again  the  TICCIT  program  seems  to  hSVe  lightened  the  burden  of  preparatory 
work. 

Before  considering  faculty  responses  to  quest^fce  about  their  general 
activities   it  may  be  useful  to  note  the  tentative  nature  of   the  data 
presented  in  Table  8.1.     Both  the  numbers  of  cases  on  which  these  data  rest 
and  the  size  of   the  standard  deviations   in  relation  to  the  means  would  be 
arguments  against  drawing  f im conclusions  from  these  descriptive  figures. 
For  example,   it  might  appear  as   if  math  instructors  at  Phoenix  had  their 
TICCIT  classes  spend  two  rjours  per  week  in  some  "other"  mode  of  instruction. 
But  the  large  standard  deviation  (4.47)  hardly  suggests  consensus  among  the 
five  respondents.     A  single  instructor  had  estimated  that  he  spent  ten 
hours  per  week  on  courseware   revisions,  and  this  one  response  led  to  the 
unrepresentative  average  given  Ln  the  table.     We  reiterate,  then,  that 
these  data  and  those  wh Lch  follow  in  this  section  should  be  viewed  as 
descriptive . 

Questionnaire  Data:     General  Activities .     instmctors'   estimates  of 
the  hours  per  week  spent  on  general  activities  cannot  be  attributed  to  the 
TICCIT  program  as  readily  as   their  estimates  ot    section  specific  tasks. 
Each   full-time  faculty  member  taught    several    classes,   and   tills  teactiing 
load  served   to  obscure  tiie    impact    of    a  single  TICCIT  class  on  general 
act  iv  1 1  iea  such   as  col  lege  comni  i  \  x  ees  ,   s  tudent^Rrganizat  Ions  ,    ini  ormal 

i 

discussions  with  colleagues  and  studonts,   and  curriculum  revision-  Yet 

\ 
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instructors  with  TICCIT  classes  reported  spending  substantially  more  time 
on  informal  discussions,  activities  to  enhance  their  own  knowledge, 
curriculum  revision  or  course  procedures,  and  the  TICCIT  project.    For  each 
of  these  general  activities  faculty  with  TICCIT  classes  estimated  devoting 
four  or  five  hours  per  week  while  faculty  with  lecture-dispussion  sections 
spent  just  one  or  two  hours  per  week. 

This  higher  level  of  general  activities  among  teachers  with  TICCIT 
classes  might  be  due  to  the  release  from  course  preparation  afforded  by  the 
TICCIT  program.     The  hours  given  to  activities  beyond  a  specific  course 
section  may  be,   in  effect,  the  counterpart  of  the  usual  preparation  time. 
This  would  be  consistent  with  the  view  that  computer-assisted  instruction 
frees  teachers  to  engage  in  other  activities-     It  seems  probable,  however, 
that  there  is  anothe     .up  .nation  for  the  increase  in  instructor's  extra- 
curricular  activity.     An  inr  vative  project  with  active  faculty  participation 
could  easily  lead  to  giuaLer  time  commitments  by  teachers-     Without  data  on 
the  total  amount  of  time  spent  on  all  teaching  >»««t  les  and  activities,  we 
fLnd  it  premature  to  choose  between  these  t^       n ten uretat ions . 

8 . 1 . b     Teacher  Tasks  under  the  TICCIT    ' rogram 

An  accurate  account  of   teacher  t  >sks  uiuU-t    the  TICCIT  program 
required  conducting  *  series  of  observation     as  iirt   ot    the  evaluation. 
These  observations   took  place  at    t  tie  demonsi        son  site  where  the  evaluator 
had  placed       Held   representative  with  prior  experience   in  observations  as 
a  mode  of  data  roLLerrimi.      The  description  of   teacher  tasks  possible  from 
divert    observations  ol    Instructors    in  TICCIT  classes  obviously  relied  less 
on   retrospective  estimates  ol    the   t  Line   spent    In  various  activities. 
It    Instead  emerged   from  close  and    regular   contact    with    Instructors  at  work 
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under  the  T1CCIT  program.    Nevertheless,  this  picture  of  teacher  tasks  is 
descriptive  only  and  does  not  represent  a  prescription  for  other  situations 
or  an  inference  about  TICCIT  effects  under  different  conditions.     It  does 
perhaps'  con t rib uVe  to  a  fuller  jppreciation  of  the  conditions  under  which  we 
conducted  this  evaluation.         /  * 

Observation  Record.     After  attempting  to  record  the  activities 
of ' all  staff  and  students  present  in  a  TICCIT  classroom,  the  observer 
began  to  focus  on  the  instructor  and  one  proctor.    Also  revised 
the  coding  scheme  with  pre-specif led  categories.     Instead  the  observer 
simply  noted  the  activity  in  which  the  instructor  or  proctor  was  engaged- 


maintained  a  record  of  the  amountf  of  time  spent  on -that  activity  within' 
a  class  period.     This  approach  proved  to  be  both  more  efficient  for. data 
collection  and  better  descriptive  of  instructor  tasks  than  a  rigid  form  for 
observations.  '  ^  .  .      .f  !/; . 

Schedule  and  Sample.     During  the  spring^nd  fall  semesters  of  the  1976 
calendar  year,  our  fielcj  representative  conducted  observations  of  math  and 
Engliah  classes  on  the  TICCIT  system.     Each  observation  lasted  one  class 
session.     In  the  spring  semester,  the  TICCIT  math  classes  at  Phoenix  met 
daily  from  7:30  am  to  3:30  pm  as  well  as  two  evenings  per  week.     Since  \ 
instructors  were  assigned  to  cover  hours  of  system  operation  rather  than  \ 


specific  course  sections,  classes  included  studSnts  ettrolled^f or  introductory 
algebra  'as  well  as  students  enrolled  for  college  algebra.     Classes  convened 
in  a  large '  room^reserved  fo'r  the  TICCIT  program;  thefroom  had  75  carrels 
tdLth  TICCIT  terminals.    On  36  occasions  the  percentage  of  time  instructors  \ 
spent  on  various  activities  was  recorded.    The  observer  also  monitored 

*  *•      ;/  '  . 

proctor  activities  in  33  of  these  observations.  I  ' 


y',  t/nl£ke*-the  TICCIT  math  -classes,  t^J^jlish  courses  offered  on^the 

TICCIT System  at  Phoenix  followed  muc^ri^fe^same  schedule  as  lecture-^. 


discussion  classesf .     Students  chose  a  particular  course  sectioi^  that  had  a 
definite  class  schedule  (three  class  perigpfc  m$r  W&5$)  as  well  as  a  specific 
instructoyV    Each  section  held  students  ^fcf igd  for  the  same  course,  and 
stu$ei*ts  always  met. with  the  same  instructors.    This  contrasts  sharply  with 
die^pyocedures  ^jj^Lh  whereby  students  registered  for  hours  on  the  TICCIT 
ays te^  rather j|         ■  a.  section  of  a  particular  course *or  classes  with  a 
specific  ins truc^r^;  However ,  English  instructors  conducted  their  classes^ 
vJE;r^e  txom  any  formal  departmental  consensus  on  teaching  with  the  TICCIT 
program  and  so  structured  their  classes  in  various  ways*  whereas  math 
f agreed  on  a^set  of  procedures  and  standards  for  all  TICCIT 

v*ir^,'-v,v^  /;.," 

*'  ■  *  'l^^g^Ses.*    ^ix' TICCIT  sections,  taught  by  four  instructors,  of  a  course 

'!• '-'    < '-  t 
./  ,  Iti-'baiic  English  were  observed  in  the  fall  semester.    Of  these  sections, 

twr6 -classes  met  regularly  in  the  same  large  TICCIT  classroom  where  math 

'Classes "met,  and  the  other  four  classes  met  in  a  smaller  room  with  about  40 

S>  ca^^ls  and  TICCIT  terminals.     The  observer  recorded  instructor  and  proctor 

r     activities  for  each  section  on  six  different  occasions,  once  every  two 

weeks, Jk>r  a  total  of  36  observations  in  TICCIT  English  classes. 

f    Context  for  Teacher  Role.     Seven  members  of  the  math  faculty  took 

assignments  to  supervise  the  TICCIT  classroom.     These 'instructors  shared 

responsibility  for  all  students  enrolled  for  math  courses  on  the  TICCIT 

,     -  -  m 

systfnn;  One  instructor  wad  present  for  every  scheduled  hour  of  system 
operation.     In  addition,  two  to  six  proctors  were  present  for  TICCIT 
classes.     The  proctors  were  students  with  a  knowledge  of  basic  math  hired 
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on  a  part-time  basis.     Nearly  20  proctora  tovered  the  classroom  hours 
scheduled  for  TICCIT  math  courses.    Typically,  one  instructor  and  three 
proctors  were  available  to  assist  students  working  on  the  computer  system. 

Class  rosters  from  the  beginning        the  semester  showed  an  average  of 
41  students  registered  for  each  TICCIT  math  class  offered  in  regular 
daytime  hours.     Counts  taken  during  observations,  however,  revealed  an  y 
average  class  attendance  of  30  students.     By  mid^semester  24%  of  the  TICCIT 
math  students'  either  had  themselves  withdrawn  from  math  courses  or  had  been 
withdrawn  tiy  instructors  .due  to  their  poor  attendance.     Indeed,  the  irregular 
attendance  patterns  depicted  in  Figure  8*2  probaJ^Ly  contributed -to  the 
failure  of  some  students  to  complete  math  courses  on  the  TICCIT  system  (see 
Chapter  4:     Course  Completion  Rates).    jThe  gabled  attendance  data  also 
verify  the  fact  that  most  students  made  a  conscious  effort  to  finish  their 
math  course  on  the  TICCIT  system,  as  suggested  by  the  number  of  hours 
*   students6  Spent  on  the  system  (see  Chapter  ,7:     Student  Activities).  -TheM 
attendance  figures  represent  thai:  average  number  $gf  students  present  in 
TICCIT  math  classes  during  five  cycles  of  observations.  Instructors 
activates  to  var?  somewhat  with  the  number  of  students  present. 

Sectic/n^of  English  courses  on  the*  TICCIT  sy&tem  had  the  same  instructor 
for  all  class  sessions.     Each  section  followed  the  procedures  established 
by  its  instructor.     Two  instructors  assigned  themes  in  addition  to  the 
writio*  requirements  already  included  in  the  flCCIT  courseware  (units  10 
and  11).     One  of  these  two  instructot^also  divided  her  students  into  four 
small  groups;  each  group  met  with  her  in  a  supplementary  conference  room 
once  a  week  for  one-half  hour.     In  these  small  group  sessions  away  from  the 

♦  - 
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TICCIT  Eastern,  students  discussed  themes  written  at  home*  ^^Astitute-  i 
took  tt\£v  instructor's  place  in  the  TICCIT  classroom  during  these  conferences. 


The  otfUr  Instructor  who  assigned  themes  gave  students  class  time  to  write  . 


their  assignments  and  discussed  the  themes  with  eacnHB^rit  individually 


during  class  time^  *A  third  instructor  also  met  with  studeajfe^'iSi  the. 

'  33$; 

conference  room,  but  on  an  individual  basis  and  primarily  about  the  writing 
assignments  from  the  TICCIT  units.     All  three  of  these  instructors  followed 
the  sequence  of  the  TICCIT  units  as  given  in  the  courseware  map  structure 
(see  Chaptei^l:     Introduction).     The  fourth  instructor  with  TICCIT'  English 
classes  in  the  fall  semester  altered  the  suggested  course  sequence  so  that 
students  worked  on  spelling  and  parts  of  speech  before  sentence  structure. 
These  changes  departed  from  the  sequence,  built  into  the  TICCIT  program,  and 
the  instructor  had  to  meet  regularly  with  individual  students  in  order 
to  keep  abreast  of  their  course  progress  and  advise  them  of  what  work 
should  be  undertaken  next. 


*£*     Each  class  period  of  a  TICCIT  English  section  al-afe^had  proctors 
and,  naturally,  students.     The  number  of  proctors  pre^^t  for  a  class 

€>e8sion^ ranged  from  one  to  four,  with  an  average  of  two  proctors  available 

*  „  j, 

r  afr  gfiven  class.    The  number  of  students  in  attendance  for  a  basic 

» 

tfaglish*  course/^ranged     from  10  to  30.    The  largest  course  section  usually  * 


had  2^students7^and ,  the  smallest  of  the  six  sections  ordinarily  had  13 
aCf ejafdanCte.    This  information  about  the  persons  present  in 
JEICClfr  .  tft&sSe's  fcnd  course  structure  perhaps  begins  to  convey  the  conditions 
undef  wt\ich  we  observed  the  tasks  performed  by  instructors. 


4  f 


Tasks  o&,  TICCIT  Teachers,    Responding  to  studlfcl  questions  about 
content  and  monitoring  student  performance  on  mastery  tests  occupied  thj^ 
bulk  of  instructors'  class  time.    Math' instructors  spent  nearly  two-thirds 
of  their^time  engaged  in  these  two  tasks,  and  English  instructors  devoted 
over  two-fifths  of  their  time  to  the  same  tasks.    Moreover,  the  five  tasks 
which  occupied  nearly  all  of  the  instructor  time  in  math  classes  also 
consumed  most  of  instructors'  time  in  English  classes.     In  order  of  the 
percentage  of  time  devoted  to  each  task,  the  five  activities'  most  of t«n 
performed  by  TKCIT  teachers  were:     answering  student  questions  on  content; 
monitoring  mastjry  tests;  circulating  among  students  so  as  to  be  alert  to 
possible  questions  or  problems;  performing  managerial  work;  and  holding 
discuss^njp  with  other  staff,  visitors,  instructors,  or  the  observer. 

*       Proctors  held  less  authority  and  responsibility  in  TICCIT  classes  and 
served  primarily  as  technicians  familiar  with  the  mechanical  aspects  of  the 
TICCIT  system  and  procedures  for  its  use.    Proctors'  time  allocations 
reflected  their ^limited  role:     they  spent  most  of  their  time  to  administer-- 
ing  off-line  tests  on  spelling.    These  proctor  functions  may  be  very  much  j 

a  function  of  the  college's  definition  df  staff  duties.    4&.  Alexandria  the 

■        .*  ■  ■  ,  • 

two  proctors  assumed  5  greater  responsibility  for  suggesting  possible  learning 
strategies  and  for  advising  students  on  conteitfr  questions .     But  one  of 
these  two  proctors  had^  taught  mathematics  at  the  secondary  school  level  and 
the  other  was  a  graduate  studeilt  in  educational  technology  at  a  nearby 
university.     The  dat|t  from  the  observations,  at  Phoenix  represent  the  tasks 
performed  by  proctors  aaji  instructor?  at  that  demonstration  site. 
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Table  8i2  presents  a  summary  of  the  percentage  of  time  which  instructors 

and  proctors  had  allocated  to  various  tasks.    It  had  not  been  the  intent  of 

the  developers  of  the  TICCIT  program  that^  instructors  spend  as  much  as  65X 

of  their  time  in.,  sewering  students'  questions  on  content  and  monitoring 
? 

test  attempts.    The  TICCI.T  program  was  to  be  self-sufficient  and  self- 
explanatory:     students  would  learn  on  the  system  and  most  students  would 
pass  mastery  tests  on  their  first  attempt.    Actually,  students  often 
sought  the  he|p  of  an  instructor  and  frequently  required  assistance 
in  their  third  and  final  attempt  at  a  mastery  test  C*ee  Chapter  7: 
Student  Activities).     Instructors  responded  to  student  needs  when  fthey  . 
provided  further  explanations  of  concepts  or  instances  presented  in  the 
TICCIT  courseware  or  when  they  guided  a  student  through  a  mastery  test. 

-      The  TICCIT  developers  had  hoped  that  their  program  would  enable 
lechers  to  adopt  a  new  role  as  manager  and  "advisor.    A  teacher  under 
he  TI&iiT  program  would  guide  students  toward  learning  objective's  and  help 
.^nCs-'to  plan  and  to  evaluate  appropriate  strategies  for  learning 
(Black,  "^^Bunderspn,  1973)  .     Seldom  could  instructors  at  the  demonstra- 
tion sites  afford' to  impend  time* on  these  activities.    Math' instructors 
devoted  just  one  percent- of  their  time  to  counseling  students  on  their  • 
^ourse  progress,  and  English  instructors  just  six  percent  of  their  time  to 
the  same  activity.     Considering  assistance  on  system  procedures  as  advice 
on  strategies  would  double  these  percentages,  but  it  still  seems  clear  thatf 
teachers  at  the  demonstration  sites  did  not  fulfill  Che  role  foreseen  by 
the  TICCIT  developers. 
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Instructor  and  Proctor  Activities  under  the  TICCIT  Program 


Tasks 


Percentage  of  Time  Engaged  In  Task 


Math 

Instructors  Proctors 


Engligfff 


Instructors  Proctors 


Answering  student  questions  on  content 
v  (explaining  material)  . 

Monitoring  tests 

(checking  answers  on  mastery  tests) 

rving  class  —  -/ 

(circulating  among  students) 

Performing  managerial  work 

(taking  attendance,  keeping  records) 

Holding  discussions  with  other  staff; 
visitors,  instructors,  pb^ery^r 

Attending  to  system  failures 
.    (fixing  student  data  areas,  consulting 
^  w4th\techiilcians) 

Assisting  students  on  system  piypfidures* 
(showing  students  how  to  j^fcTystem) 

Counseling  students 
'(advising  student^,  on  course  progress) 

Gtadlng  assignments 

(reading  and  marking  student  themes) 

jAditf^a£ering  off-line  tests 

(d#4^.ing  students  in  Spelling) 

FulKiljLt^g;Thiacellaneo,i49  duties 
(making  announdfepients ,  etc.) 

Working  at  terminal 

•  :  / 

Not 'present,  in  cl&ssroosT^ 


Number  of  observations 


34 


31 


10 


-  8 


1 


1-  ^ 


43 


16 


20 


4 


8m„ 


29 


15 


10 


6 
6 
2 


36 


28 

10 
\ 


28 


20 

•2i 

1 

6 


36 


Table  8.2 
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To  a  large' extent,  ^rfstructprs  under  the  TICCIT  program  performed 

thosa  tasks  demanded  by  tl>e%  situation.  Students  often  required  further 

explanation  of  the  material,  available  on  thJ-TICCIT  system.     It  was  common, 

especially  In  math  .classes  where  there  was  a  high  student-teacher  ratio,  to 

see  a  teacher  working  with  a  student  on  a  third-test  attempt  while  ^her 

studeal|Nfaited  for  assistance* The  pattern  of correlations  between 

nutfber  of  students  present  and  teacher  time  spent  on  various  tasks  ^inforce 

this  point.     Instructors  tended  to  devote  more  time  to  answering  student 

questions  on  content,  monitoring  tests,  anfr  performing  managerial  tasks 

jfien  larger  numbers  of  students  were  in  attendance  (r  -  .14,   .15  and  .25, 

respectively,  for  mainline  TICCIT  math  classes^.    And  they  spent  less  time 

counseling  students  and  assisting  students  on  system  procedures  as  attend-'  ^ 

ance  increased  (r  -  -.26  and  -.22,  respectively).    Similarly,  p^tors  sp«nt 

greater  amounts^  time  on  assisting  ^students  in  the  m^anical  detail  of 

system  use  when  more  students  attended  their  TICCIT  math  classes  (r  * 

It  may  be  that  the  TIC&IT  program  was  sufficient  to  support  independent 

student  study  but  that  neither  faculty  nor  students  could  adant  to  this  new 

learning  environment.     After  years  of  instruction  delivered  in  a  conventional 

manner,  students' perhaps  found  themselves  unable ^t<?lake  full  advantage  of 

tha  learner  control  offered  on  the  TICCIT  system.    Certain  pe*:entages  pf 

<* 

students  did  complete  courses  on  the  TICCIT  system,  attain  higher  scores 

\  ' 

on  achievement^posttests  than  comparable  students  in  lec'ture-discussion  % 
sections,  and  they  did  so  in  less  time  than  would  ordinarily  be  required 
f£>/ attending  classroom  lectures  and  discussions.    Most  community  college^ 
students  enrolled  in  courses  on  the  TICCIT  system,  however,  simply  failed 
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to  complete  the  requirements  for  course  credit.    Perhaps  Instructors 
trained  to  serve  as  a  subject  matter  resource  persisted  In  that  role  with 
the  TICC1T  program  and  failed  to  provide  the  counsel  and  advice  on  learning 
strategies  which  may  have  been  necessary  for  all  students  to  benefit. from 
the  TICCIT  program.     Perhaps  Instructors  with  special  skills  In  promoting 
effective  learning  strategies  on  Instructional  systems  such  as  thev\lCCIT 
system  could  help  the  program  to  reach  a  greater  proportion  of,  students. 
The  strongest  evidence  for  such  a  .position,  would  be  a  counterexample  to  the 
rasults  of  this  demonstration* 


8.2    Faculty  Acceptance   *  ;  sr  ^  - 

t  In  their  recent  review  of  the  literature  on  teacher  attitudes,  Stern 

and  Keislar  (1977)  concluded  that  teacher  attitudes  do  make  a  difference 
in  the  teaching-learning  process  and^that  tocher  attitudes  can  be  altera^. 
Both  of  these  conclusions  bear  implications  for  innovative  educational 
^  programs  such  as  the  TICCIT  proj^|t.    The  outcomes  obtained  with  a  program 
may  be  attributed,  at  least  in  part,  to  instructor  attitudes  toward  that 
program.     Strong  teacher  resistance  could  sabotage  another  vise  effective 
project  or  high  teacher  enthusiasm  might  be  the  real  explanation  for 
observed  positive  outcomes.    Moreover,  a  program  itself  may  Ifead  to  changes 
in  teacher  attitudes.    Demonstrated  improvements  in  student  performance, 
for  exampl^,  might  be  expected  to  generate  teacher  support  for  a  program. 
To  document  the  initial  receptivity  of  faculty  and  their  subsequent  reactions 
to  the  TICCIT  program,  assessing  teacher  attitudes  be'CarniTft  part  of  the 
evaluation  for  the  TICCIT  project.  \ 


»  ■ 
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It  has  been  suggested  that  the  resistance  of  classroom  teachers 
represents  the  greatest  single  Impediment  to  the  widespread  implemen- 
tation of  instructional  technology  in  our  schools  (Armsey  and  Dahl,  1973). 
Anastasio  and  Morgan  (1972)  offered  several  plausible  explanations  for  this 
resistance  as  it  applies  to  computer-assisted  instruction:    fear  of  the 
changes  which  result  from  a  technology  as  complex  as  the  computer;  ignorance 
of  the  computer's  potential  and  limitations;  and  the  clash  of  values  that  A 
arises  from  the  teacher's'  perception  of  the  computer  as  a  replacement  for 
personal  interactions  with  students.     Tobias  (1968)  had  earlier  noted  that 
the  introduction 'of  automated  devices  into  schools  poses  s  threat  to 
teachers,  snd  others  (e.C,  Olivier,  McLean,  Brahan  &  Payne,  197^  have        \  > 
stressed  the  importsnce  of " teacher  attitudes  in  determining  the  impact  of 
innovative  curriculsr  programs  based  on  computers^^ 

If  'tW  TICCIT  program  succeeded  in  fostering  positive  fsculty  attitudes, 
this  slone  could  very  well  constitute  a  significant  contribution  to  the 
field  of  instructional  technology,     it  would  also  reflect  the  positive 
judgment  of  those  closest  to  the  project,  instructors  and  administrators  "at 
the  demonstration  sites,  on  the  quality  of  the  program. 

8. 2. a    Baseline  Attitudes  *  ' 

Assessing  chan^^a.*Wt:lt»»4fis  and  understanding  initial  rec*|- 
tivity  required  a  survey  of "faculty  attitudes  conducted  prior  to  the  implemen- 
tation of  the  TICCIT  program.     Such  a  survey  was  conducted  in  the  early 
stages  of  the  project  (Alderman  &  Mahler,  1977).     However_J the  response 

r 

rate  for  this  survey  was  low  as  faculty  completed  and  returned  only  one- 
*    third*  of  the  total  number  of  forms  distributed.    The  conclusion*  drawn  from 
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that  initial  survey  were  therefore  tentative  and  perhaps  unrepresentative 
of  the  colleges'  beliefs  about  educational  practices  and  their  receptivity 
toward  computer-assisted  instruction.    Still,  the  initial  faculty  survey 
helped  to  focus  later  q[uestio^s  on  those  aspects  of  educational  practices 


ttrtate£  to  th* l '  ppemlmfif  Of  t*W  T;iCCIT  program  and  to  improve  procedures  for 
eelftcHtig  items  and  distributing  questionnaires.  > 

A  second  faculty  survey"pT£ceded  the  introduction  of  the  TICCIT 
system  into  the  community  colleges.     This  survey  took  place  in  the  spring 
term  of  the  1973-74  academic}  year;  the  TICCIT  system  was  installed  in  the 
summer  term  of  the  same  academic  year.    The  number  of  questions  included  on 
the  survey  was  less  than  one-half  of  the  number  on  the  original  form.  And 
the  response  rate  was  over  twice  that  obtained  with  the  initial  survey.  At 
the  time  of  thU  second  survey,  faculty  had  attended  general  orientation 
sessions  about  the  TICCIT  program,  and  some  had  participated  in  specifying 
content  coverage  and  planning  for  the  demonstration.    Administrators  and 
*  instructors  certainly  knew  more  about  the  TICCIT  project  and  perhaps  were 
better  able  to  appreciate  the  intent  of  the  questions  on  the  survey.  A 
questionnaire  specifically  focused  on  mat|ers  related  to  the  TICCIT  program, 
the  further  refinement  of  a  tried  and  protjh  version  of  a  faculty  attitude 


survey,  college  faculty  informed  about  the  TICCIT  project,  questionnaire 
distribution  prior  to  the  installation  o^  TICCIT  systems,  arid;  a  high  * 
response  rate,  all  of  these  factors  provide  reason  to  accept  the  results 
which  follow  as  representative  of  initial  faculty  attitudes. 


Inatrumcnt.  *The  survey,  form  (see  Appendix  I,  Form  02T)  was  a  ref ine- 
Mnt  of  the  original  attitude  questionnaire.    It  contained  essentially  two 
sections:    the  first  dealt  with  assumptions  about  educational  practices 
implicit  in  the  design  ^f  the  TICCIT  program  and  the  second  concentrated  on 
poAar  beliefs  about  computer-assisted  Instruction.    Items  in  the  first 
section  sampled  opinions  about  course  objectives  (items  1-3),  curriculum 
structure  Ut%m  4-7),  student  management  (items  8-11),  practices  for 
grading  and  testing  (items  12-15),  and  institutional  receptivity  to 
innovation  ( item*  16-17) .    Statements  in  the  second  section  concerned 
faculty  familiarity  with,  computers  (items  18-19),  the  impact  of  CAI  on  the 
teacher's  role  (items  24,  25,  30),  CAI  potential  for  student. learning 
(items  21*23,  26,  27),  and  global  judgments  of  CAl's  place  in  education 
(items  20,  29).    Throughout  the  instrument  there  was  emphasis  on  the 
learner  control  and  mastery  testing  so  important  in  the  design  of  the 
TICCIT  program  (see  Bunderson,  1973). 

Each  item  on  the  survey  was  a  statement  accompanied  by  a  five-point 
response  scale  for  indicating  extent  of  agreement.     So  as  to  inhibit  the 
formation  of  response  sets,  there  were  statements  phrased  in  Dotfra  positive 
(e.g.,  "CAI  tailors  instruction  to  the  indiyidual  student.")  frd  negstive 
(e.g.,  "CAI  is  s  potential  threat  to. the  jobs  of  faculty  members.")  manner. 
The  scores  assigned  to  responses  ranged  from  five  for  strong  agreement  with,, 
the  statement  to  one  for  strong  disagreement.     In  addition  to  these  items, 
there  wss  one  question  sbout  fsculty  priorities  in  the  evslustion  of  CAI 
(item  31)  end  snother  about  their  knowledge  of  the  TICCIT  program  (item  32). 
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Res  ponss  Rate.    The  survey  aleng  with  a  letter  of  explanation  and  a 


j^splyjlnvelope  was  mailed  directly  to  the  314  faculty  members  at  the  colleges. 
^A|T#i  fo.ur  weeks  letters  were  sent  to  97  faculty  members  who  h^i  not  yet 
responded.    Within  seven  weeks  239  completed  forms  had  hey  returned  for  a 
response  rate  of  76Z.    Undoubtedly,  the  shortenfefl  length  of  the  survey  and  / 
the  anonymity  of  the  fci^Ptctotr ibuted  to  the  high  rate  of  return.  , 

The  survey  did  require  inf onpat ion  about  each  respondent's  college, 
•  division,  depart meat,  and  position.    And  ther%  was  some  fluctuation  in  the 
response  rate  aMH  ■beta -categories.     For  example,  the  response  rate  from 
Alexandria  was  nj  from  Phoenix  79Z;  math  instructors  returned  83Z  of 

tlwfc  forms  Aent  ttrtSm  vhile  English  instructors  returned  7 IX.     But  the 
rate  for  any  gtXjgA  category  of  faculty  members  tended  to  be  close  to  the 


average,  as be  expected  from  the  high  response  rate  overall.  Since 


over  three-fburths  of  the  faculty  responded  to  the  survey,  the  data  should 
yield  a  fair  description  of  faculty  opinions  about  educational  practices 
and  of  faculty  beliefs  about  computer-assisted ^Instruction •    The  candor  of 
the  comments  made  on  the  survey  furtherr  suggest  that  this  description  is 
accurate  as  well  as  fair.  v 

*  Analysis.     Much  of  the  richness  in  these  data  can  be  found  at  the  item 
level.     Simple  descriptive  statistics,  namely  the  mean  and  standard  deviation 
of  the  responses  to  an  item,  suffice  for  revealing  the  point  of  consensus 
and  extent  of  variation  in  faculty  attitudes  on  a  particular  matter.  YeV 
it  is' also  helpful  to  consider  the  dimensions  underlying  these  responses 
to  individual  Items.    The  pattern  of  responses  can  reveal  the^major  constructs 


upon  which  faculty  base  their  opinions.    Thus,  the  results  of  the  survey 

will  be  presented  through  a  factor  analysis  of  the  data  and  thrpugh  descrip- 

J 

jtive  statistics  for  each  item., 

Separate  factor  analyses  were  performed  for  the  \fwo  sections  of  the 
survey.    Previous  results  had  indicated  that  faculty  opinions  about  educa- 
tional practices  and  their  attitudes  toward  computer-assisted  instruction 
involved  independent  affective  dimensions  (see  Alderman  &  Mahler,  1977). 
In  other  words,  the  basic  structure  behind  faculty  responses  to  statements 
about  educational  practices  overlapped  little  with  the  pattern  of  responses 
to  statements  about  computer-assisted  instruction.    The  dimensions  depicted 
by  factor  analysis  would  thus  be  nearly  the  same  whether  the  analysis  was 
conducted  for  both  sections  together  or  for  each  section  by  itself.  Because 
the  two  sections  concerned  two  quite  different  subjects,  we  preferred  to 
conduct  separate  factor  analyses. 

Of  the  total  number  of  239  forms  returned  by  faculty,  there  were  215 
with  a  response  to  every  statement  in  the  first  section  of  the  survey  and 
223  with  a  response  to  every  statement  in  the  second  section.     The  item 
correlation  matrix  for  the  17  statements  on  educational  practices  was 
therefore  based  on  the  data  from  215  respondents.     Similarly,  the  item 
correlation  matrix  for  the  13  statements  about  computer-assisted  instruction 
was  based  oh  the  data  from  223  respondents.     Each  of  these  item  correlation 
matrices  was  subject  to  a  principal  components  analysis  with  varimax 
orthogonal  rotations.  ( 

Beliefs  about  Educational  Practices.     Four  major  factors  emerged  from 
the  principal  components  analysis  of  the  first  section  of  the  survey.  The 


,   eigenvalues  fpr  tRese  four  factors  were  2.82-,  2.35,   1.59  and  1.21.  Since 
eigenvalues »  in  retat^ipn  to  the  total  number  of,  items  give  the  amount  pf 
variance  accounted  for  by  a  factor  analysis,  the  four  factors  accounted  for 
47%  of  the  total^ variance  in  the  item  correlation  matrix.    The  vafimax 
solution  itself  is  shown  in^Table  8.3.     Most  items*  clearly  loaded  under 

Tjust  one  factor:    an  item's  highest  loading  came  under  one  factor  with  much 
lower  loadings  elsewhere.     For  example,  the  first  item  had  a  loading  of  .69 

.  under  Factor  II  and  loadings  of  -«Q3,  -,03  and   .12  under  the  other  three 
factors.     And  the  seventeenth  item  had  a  loading  of  -.82  under  Factor  IV 
with  much  lower  loadings,   ,04,   .13  and  .02 ,  AimieT~^ti^r~f^Kite^s .  The^jfew 
exceptions  to  this  pattern  (i.e.,  items  2,  J  and ^14)  do  seem  to  belong 
under  multiple  factors.     The  general  factor  structure  given  in  Table  8.3 
suggests  four  dimensions  for  describing  faculty  beliefs  about  educational 
practices.     According  to  the  statements  under  eacj^  factor  (shown  in  Table 
8.4),  we've  chosen,  to  label  the  four  dimensions*  Stu4ent  Focus ,  Subject 
Matter  Emphasis,  Receptivity  to  Innovation,  and  Incentives  for  Student-  Work. 

The  first  factor  accounts  for  the  most  variance  and  seems  to  reiterate 
the  importance  of  a  focus  on  students  af  community ^colleges •     Because  the 
statements  under  this  factor,  as  listed  in  Table  8.4,  suggest  the  student 
is  the  focal  point  far  curr^cular  decisions  and  interpersonal  contacts , 
Factor  I  has  been  labeled  Student  Focus.     The  items  most  prominent  tinder 
this  factor  concern  discussions  among  students  and  informal  interactions 
between  students  and  faculty.     These  items  (items  7  and  11)  were  among 
those  with  the  highest  mean  and  smallest  standard  deviation:  * 
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Vanity  Attitudes-  toward  Educational  Practices 
Rotated  Factor  Matrix 
(after  varimax  orthogonal  rotation) 
/  Alexandria  and  Phoenix  • 

-  Spring  -1^74. 


factor 

i 

i Loadings 

,Item  ' 
Number 

I 

■  TT 

TT  T 
XXX 

•4 

IV 

-  .03 

'.69 

-.03 

'  .12 

■> 

3  2 

■  .Al 
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,;.    '  3 
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'•  .36 

.09 
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■°  .56 
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.12 

5 

^  .50 

.26 

.32 

.03 

6 

.56 
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.08 
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.75^ 

-.03 
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'  .20 
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-.04- 
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.73 
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.11 

■  .71 
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13 

.43, 
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.22 
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-.10 
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15 

-T.14 
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•  i  .io 

-.S32 
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16 

.04 
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.04 
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V 
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\1  Faculty  Attitudes  toward  Educational;  Practices - 
f.v'?         '  '  Varimax  Primary  Factors 

n  1      ,  '  .  Alexandria  and  Phoenix 

U  '  "  Spring  1974    ,5  u 

41  :  — *  


Item 
Numb  e  r 


Stem 


Factor  * 
Loading  Mean 


sd 


Factor  I: 
■    7  . 
11 

10 


Student 


F^cus 


Discussions^  among  students  contribute  to  their  learning 

4~  '  ■ 

Informal  interactions  between  ftudents  and  faculty  are 
an. important  part  of  education.  V 

Students  can  benefit  from  increased  flexibility  and 
responsibility  for  their  own  instruction. 

College  instruction  should  allow  each  student" to  proceed 
at  his/her  own  pace. 


1 


g/le 


13 

;  V.- 
2 


Factor  II: 
12 


1 


Student  feedback'  is  essential  in  preparingymew  course 
material. 

/ 

A  student's  progress  is  more  important  than  his  final  ' 
level  of  achievement.  ■  J 

5 

> The  development  of  self-confidence  and  a-,  sense  of  accom- 
plishment should  be  an  essential  part  of  every  course. 

K  ' 

College  Courses  must  develop  students1;  interest  in  and 
appreciation  of  tjie  sujx>ect.  ^        -  m 

'    „  *       ,  *  j 

Subject  Matter  Emphasis 

Students  shqulcti  be  evaluated  against  a  well-defined 
criterion  of  knowledge  ejr  skills, 

'  i    '  ,. 

There  are  specific  concepts 'or  skills  which  students 
in  all  sections  of  a  course  must  master. 

Testing  is  an  important  and  integral  part  of  the 
educat ional  process . 

The  primary  bas£s  for  the  organization  of  a  course-" 
.  should  be  the  intrinsic  organization  of  the  subject 
matter .  - 

The  development  of  self-confidence  and  aJ sense  of  accom- 
plishment should  be  an  essential  part  of  every  course. 


Factor  III: 
16 

17 

Factor  IV: 
8 
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Receptivity  to  Innovation 

It  has  not  been  possible  for  rjew  ideas  about  educational 
practice  to. receive  a  hearing  at  this  institution. 

Administrators'^  ^'partmenL  chairmen  generally  encourage 
faculty  to  experiments wftC^new  courses  and/or  teaching 
methods. 

Incentives  for  Student jWork 

Without  close  fa^ulty^supervision ,  many  students  at  this 
college  would  not  be  able  to  sustain  sufficient  tnotiva- 
t ion  to  complete  their  stud ies .  V 

Most  students  need  peer  competition  as  an  incentive)  tojc 
working  an<y  learning:  ./,■*' 

The  response/oil i t'y  for  a?  student '  s  grade  must  rest  \" 
with  the  instructor.       .  /  '  . 


CK^-  215) 
*  75         41; 50,  .65 
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.50 
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.41 


.39 


.71 


.69 


.57 
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.56 


.44 


.84 


.82 


^3 


.60 


.49 


4.45 


.57 


3.87  '.80 


3.61  1.00 


4.34  ' 


.75 


3.44  1.05 


4.64 


4M7 


.56 


.86 


4.00  .85 


4.61 


4  .00 


.64 


.85 


/ 


3.60  .25^ 


4.64  .56 


1.78  .89 


~3.99  .81 


4.05  .,91 


3.31 


.97 


3^33  1.26, 


/ 


Tsfcle  8. A 
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faculty  stressed  the  role  of  interpersonal  contacts  in  teaching  community  ^ 
college  students.    Ye,t  computer-assisted  instruction-might  threaten  just 
such  discussions  and  '-informal  interactions .  *"*The  TICCIT  program  did  promise-  v 
students  greater  involvement  in  their  own  learning  and  an  opportunity  for 
instruction  geared  to  each  individual's  rate  of  learning.     Such  goals  would  ( 
be  consistent  with  a  student  focus  (items  10  and  6  under1  Factor  I  in  Table 
8.4) ,  but  faculty  endorsement  of  these  practices  is  lower  than  the  importance 
attached  to  interpersonal  contact^.    Fa^cQlty  also  concurred  that  courses 
-should  develop  students'   self-confidence  and  their  appreciation  of  the  ;* 
^ubjec't  matter  (items  2  and  3  in  Table  8.4).     These,  too,  were  among  the/' 
goals  of  the  TIQCIT  program.     However,  a  common  criticism  of  computer-assisted 
-  iris  truct  ion  in  general  has  been  expressed  as. doubt  of  its  potential  for 
positive  impact  on  the  affective  aspects  of  student  learning.  Despite^ 
the  consistency  of  the  goals  of  the  TICCIT  program  with  a  student  focus, 
there  would  se(em  to  be  possible  points  of  conflict  with  faculty  beliefs 

I 

about  educational  practices.  % 

The  statements  grouped  under  Factor  II  deal  with  several,  of  the 
premises  of  the  mastery  learning  scheme  inherent  to  the  TICCIT>program  and 
reflect  on  emphasis  on  the  subject  matter  of  a  course-    The^e  statements" ' 
(items  12,   I,   14  and  4  under  Factor  11^  in  Table  8.4)  repeated  basic  tenets 
of  mastery  learning:     the  use  of  a\ well-defined  criterion  of  knowledge  or 
skills  for  evaluating  students;  trie  importance  of  students  mastering 
^specific  concept^;  the  role  of  testing  in,  learning;  and  the  structure  of 
'    the  subject  matter  as  *he  principle  basis^or  course  organization.  The 

§ 

TICCIT  program  depended  on  these  tenets,  and  faculty  seemed  to  believe  in 
them  as  sound  practices.     The  second  item  on"  the 'survey,  already  reported 
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■  • 

under  Factor  I,  appears  again' alongside  statements  about  student  mastery 
of  the  subject  matter •     This  suggests  both  the  awareness  of  faculty  about 
mastery  learning  and  the^strategy  by  which  the  developers  of  the  TICCIT 

program  hoped  to  promote  self-confidence  and  a:  sense  of  accQmplishment 

4?* 

among  students.     The  factor  itself  seems  to  represent  the , counterpart  of 
a  Student  Focus  as  ^fc^rtresses  the  importance  of  the  subject  matter  in 


oft  facul 


learning . 

C 

The  third  dimetiBion  c£  faculty  opinions  about  educational  practices  is 
Recept^v*Ity  to  Innovation.    The  two  items  under  this  factor  were  adapted 
from  another  instrument  (Peterson,  Centra,  Hartnet  &  Linn,   1970)  specifi- 
cally If  or  their  emphasis  on  a  college's  openness  to  change.     It  was  to  be 
expected  that  colleges  receptive  to  the  TICCIT  program  also  would  be  open 
to  new  iqea8  about  educational  practice  (item  16)  and  would  encourage 
faculty  to  experiment  with  new  courses  and  teaching  methods  (it^m  17). 

*  1  <v 

Over  eighty  percent  of  the  faculty  members  who  responded  to  -the  survey  (see 


Appendix  T  for  response  distributions  by  item)  perceived  jtheir  institution 
as  receptive  to  innovations.     In  this  respkctithe  two  community  colleges 


constituted  fertile  ground  foT  a  demonstration  of  the  TICCIT  progjflam. 

Factor  IV  contains  statements  about  student  management.  Faculty 

'    4  -  .    ■  '    -  ^y 

supervision  (item  8),  peer  competition  (item  9)  and  grades  (item  15) 

*  -i  ?•  >■ 

might  all'be  viewed  as  incentives  for  stfttient  work.     Faculty  agreed  * 

_>>  if 

that  close  supervision  helps  students  by  austki^iing  ^  their  motivation 

to  complete  their  students.     There  was  far  le^e  agreement  about  the  need 

*    .  ar 

rfor  .peer  competition  among  students  and  tl^, responsibility  for  student 
grades.     The  importance  associated  with  close  faculty  supervision  reiterates 
the  emphasis  placed  on  interpersonal  contacts  in  student  learning  (items  7 


and  11  under  Factor  I).     And,  again,  tliis  faculty  opinion  suggests  a 
possible  conflict  should  the  TICCIT  program  supplant  instructors.     If  the 
TICCIT  program  eliminated  peer  competition  or  automatically  assigned 
student  grades,  neatly  one-fourth  of  the  faculty  might  welcome  such  changes 

Jther  fourth  would  at  least  voice  no  objections   (see.  Appendix  T) . 
e  generaLockscription  of  faculty  attitudes ^oward  educational 

♦r* 

practices,  as  given  abovfe  and  supplemented  by  the  descriptive,  statistics  in 
'  <*■ 

Table  8.4  and  the  iteni  response  distributions  in  Appendix  T,  seeps  to  hold 


anc 


across  ac* 


ademic  divisions  and  pos it i^n^N  Table  §.3^^eent^  -descriptive 
statistics  at  the  item  l^vel  for  all  respondent^  jjjjr  Academic  divisions, 
and  for^fnstructors  and  administrators.    'The  1^ em  means  for  all  respondents 


stay  within  five-hundred ths  >of  a  point        the  respective  means  for  respondents 
with^cbmplete  data  in  the*fiijfet  ^sec&on  off  the, survey.     Faculty  attitudes 
^lsb  appear  to  be  consistenf|^ossv division^  or  positidns:     it  tern  means 


/Var^'  little,  across  these;  categori.^w    Hc^ve^,  administrators  did  endorse 

Xiit 


0  \^ 

self-paced  instruction  /(  it  em/ 6 ).  liid  definite  Criteria  for  evaluating 

students  (item  12)  more/. strong lV  t^'a£  instructors  hatfT"  And  faculty 

r  i'      A   4  /  •  . 

Members  from  the  natural  ^sciences  seamed  tojjjf&ce  greater  emphasis  on  the 

subject  matter  (items  1^, ^  and  4)  than  lid  their  colleagues  from  other 

t disciplines.     Indeed,  th^s  dimension  reflected  the  widest  discrepancies 

amoti^faculty^divisions  in  their  opinions  about  educational  practices.  But 

even  where  dif ferences^of  opinion  existed^among  the  major  academic  divisions 

.  \- 

or  between  instructors/and  administrators,  t^ese  differences  involved  the 
extent  of  their  agreem^t  on  a  matter  rather ^than  the  advocacy  of  opposite 
extremes.  3 


\ 


/  Faculty  Attitudes  toward  Educational  Practices 
'    Responses  across  divisions  and  positions 
Alexandria  and  Phoenix 
'  S^ing  1974  i 

V.  \ 


.  \    '  *    <  ' 

Item 
Number 

All 

Respondents . 
(N  *  239) 

Hean  sd 

Natural 
Sciences 
(N  a  38) 

Mean  sd 

Social 
Sciences 
(K  a  24) 

Mean    sd,,  ■ 

Humanities 
(N  a  58) 

Mean  sd 

r 

Applied 
Studies 
(N  a  66) 

Mean  sd 

Instructors 
(N  2  170) 

Mean  sd 

Adminis- 
trators 
(N  ar  29) 

Mean  sd 

7 

•11 

\             ,.  10 
6 

Factor  I:  5 
Student  Focus  13 

2 
3 

4.50  .65 
4.44  .59 
3.88  ;  .81 
3.62  1.00 
4.32  .77 
3.48  1.05' 
4.65  .56 
4.17  .87 

4.49  .64 
4.56  .50 
3.87  ,65 
3.55  .94 
4.33  .80 
3.24  1.09 
4.66  .57 
4.23  .92 

4.38  .81 
4.36  .69 
3.96  .92 
3.62  .79 
4.12  1.07 
3.54  1.01 
4.46  .76 
4,42.  ,81 

4.64  '  -.58 
4.44  .59 

0  0/  01 

3.84  .83 
3.55  .93 
4.25  ,73 
3.64  .96 
4.69  .50 
4.20  .93 

4.34  .68 
4.34  .58 
3.79  ,8/ 
3.41  1,15 
4.28  ,77 
3.41  1.17 
4.68  .53 
4.01  ,88 

4.46  .69 
4.42  .59 
3.80  .83 

3.53  1,02 
4,28  .80 

3 . 54  1.05 
V.62  .59 
4.17  .87 

4.66  .54 
4.62  .49 
4.14  ,73 
4.07  .74 
4.38  .81' 
3.45  1.04 
4,76  .43 
4.10  .84 

•    '  12 
1 

Factor  II:    .  ^ 
Subject  Matter  4 
Emphasis     |  2 

3.98  .88 
4.60  .66 
4.00  .83 
3.60  .99 
4.65  .56 

4.28  .71 
4,34  .36 
4.36  .58 
3.82  .94 
4.66  .57 

3.56  1.20 
4.38  .70 
4.12'  .85 
3.50  1.25 
Mfcf  .76 

3.95  .89 
4.47  .75 
3.95  .86 
3.84  .83 
4.69  .50 

4.01  .81 
4.71  .57 
3.93  .85 
3.65  .95 
4.68  .53 

3.95  .89 
4.60  .66 
4.01  .81 
3.63  ,96 
4.62,  .59 

[  1 

if  ' 

4.31  .65 
4.65  .66 
4.10  .88 
3.41  1.19  ■ 
4.76  .43 

Factor  III:  » 
Receptivity  to 

T  * 

1  .  Innovation         *  yj 

1.77  .87 
3.99  .80 

1.64  .89 
4.13  .85 

2.12  1.05 
3.69  .82 

1.74  .96 
4.03  ,85 

1.67  ,  .64 
4,17  .69 

■  1.79  JO 
3.99  .82 

1.59  .6(2, 
4.03  .56 

,.  Factor  IV:  8 
Incentives  for  9 
Student  Work         1  15 

4.03  .91 
3.29  1.01 
3.29  1.27 

4.11'  .82 
3.59  .81 
3.67  1.37 

4.21"  .87 
2.92  1,14 
3.71  1,10 

4.17  .94 
3.56  ,89 
3.05  1.21 

4.00  .82 
3.15  1.00 
3.31  1.31 

4.07  .89 
3.36  .99 
3.32  1.27 

4.07  .91 
3.17  1.18 
3.21  1.32 

er|c  484 


Table  8,5 
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On  the- whole,  faculty  responses  to  the  first  section  of  the  survey 
indicated  attitudes ^s.upportive  of  the  premises  inherent  to.  the  design  of 
the  TICCIT  program.     Faculty  generally  agreed  with  statements  reflecting 
the  baaic/teuets  of  mastery  learning,  and" the  mastery  tests  and  hierarchical 
content  structure  adopted  for  the  TICCIT  program  incorporate  design  features 
of  mastery  learning.     As  viewed  by  their  faculty,  the  colleges  selected  as 
demonstration  sites  for  the  TICCIT  project  were  receptive  to  educational 
and  curricular  innovation.     The  colleges'  active  participation  in  the 
'project  certainly  confirms  this  view.     Since  most  faculty  did  not  perceive 
peer  competition  as  essential  to  student  learning  and  did  not  see  grades  as 
a  necessary  responsibility  of  the  instructed ,  there  was  leeway  for  the 
TICCIT  program  in  assuming  a  role  in  student  management. 

The  clearest  and  most  prominent  dimension  behind  faculty  attitudes 
toward  educational  practices  was  a  focus /on  the  interests  of  the  student. 
Faculty  seemed  to  base  their  responses  on  what  they  considered  best$}for  the 
student.     And  they  certainly  believed  in  the  importance  of  personal  inter- 
actions for  student  learning.     Over  90%  of  the  faculty  agreed  that  discussions 
among  students  and  informal  interactions  between  students  and  instructors 
contributed  to  the  educational  process.     A  clash  of  values  might  arise  if 
the  TICCIT  program  inhibited  such  interactions,  or  faculty  might  readily 
accept  the  TICCIT  program  if  it  instead  facilitated  interpersonal  contacts. 
This  dimension  of  Student  Focus  had  also  appeared  in  the  factor  analysis 
conducted  for  the  original  faculty  attitude  survey  (Alderman  &  Mahler, 
1977),  and  a  point  made  in  that  earlier  .report  perhaps  warrants  repetition 
here,  ^ften  innovative  programs  attempt  to  exploit  the  unique  potential  o 
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a  particular  delivery  mechanism,  yet  faculty  acceptance  would  seem  to 
depend  on  the  extent  to  which  a  program  addresses  student  qeeds  and  perhaps 

•>■ 

takes  advantage  of  a  student's  unique  potential  for  leatning.     The  TICC 
program  did  seek  to  exploit  each  student's  capacity  for  learning  through 
its  leajcner  control  and  its  mastery  approach  to  instruction. 

Attitudes  toward  CAI .     In  contrast  to  the  four  factors  found  to  i 
underlie  faculty  beliefs  about  educational  practices,  two  factors  sufficed 
for  describing  faculty  attitudes  toward  computer-assisted  instruction.  A 

'  J 

break  in  the^magnitude  of  latent  roots  after  the  second  root  had  determined 
the  number  of;  factors  to  be  rotated.     The  eigenvalues  associated  with  these 
two  factors  were  5.24  and  1.54.     Thus,  the  varimax  solution  shown  in  Table 
8.6  accounted  for  52%  of  the  total  variance.     The  two  dimensions  to 

this  solution  reflect  general  attitudes  toward  CAI  and  familiarity  with 

\ 

cQmputers . 

Items  that  dealt  with  CAI  (items  20-30)  loaded  under  Factor  I  in  the 
principal  components  analysis  conducted  for  the  'second  section  of  the 
survey.     Unlike  items  from  the  first  section  of  the'  survey  which  had  means 
near  the  extremes  of  the  response  scale  (above  four  or  below  two  on  the 
ftve-point  response  scale),  all  of  the  statements  in  the  second  section 
resulted  in  means  near  the  scale's  mid-point  of  the  scale.     Table  8.7 
presents  the  item  stems  along  with  their  respective  factor  loadings, 
means,  and  standard  deviations.     The  only  statements  with  which  a  majority 
of  ,the  faculty  agreed  were  those  about  the  strength  of  the  immediate 
feedback  available  with  CAI  (item  26),  the  potential  for  CAI  to  relieve 
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Faculty  Attitudes  toward  CAI 
Rotated  Factor  Matrix 
(after  varimax  orthogonal  rotation) 
Alexandria  and  Phoenix 
Spring  1974 


Item 
Number 

Factor 
I 

Loadings 
II 

18 

.07 

.91 

19 

.14 

.91 

20 

.64 

.28 

21 

.69 

.10 

* 

22 

-.55 

'  -.07 

23, 

.72 

.02 

24 

-.31 

.05 

25 

.64 

.16 

26 

.74 

.15 

-  27 

.79 

.08 

28 

-.63 

-.06 

29 

'  -.73 

-.15 

30 

.76 

.20  . 

Table  8.6 
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Faculty  Attitudes  toward  CAI 
Varimax  Primary  Factors 
Alexandria  and  Phoenix 
Spring  1974 

9 '  . . 

Item 
Number 

* 

Stem 

Factor 
Loading  Mean 

fid 

Factor  I: 

General  Attitudes  toward  CAI 

(N 

a 

223) 

27 

CAI  will  make  students  more  active  agents  in  their 
o\*m  education. 

.79 

3. 

57 

.79 

30 

26  \ 

**        «*  ) 

CAI  can  make  better  and  fuller  use  of  instructors 
capabilities.  \ 

Immediate  feedback  to  students  makes  CAI  a  highly 
desirable  instructional  method.* 

.76 

,    %74  ; 

3. 
3. 

58 
90 

.82 
.74 

29 

CAI  is  a  passing  fad. 

-.73 

2. 

46 

.77 

23 

CAI  tailors  instruction  to  the  individual  student. 

.72 

3. 

40 

.88 

21 

CAI  allows  students  to  assume  greater  responsibility 
for  their  own  learning. 

.69 

3. 

60 

.80 

20 

CAI  is  one  of  the  mo/st  significant  developments  in 
education  today. 

.64 

3. 

15 

.90 

25 

CAI  can  relieve  instructors  of  routine  duties. 

.64 

3. 

70 

.92 

28 

Student  interest  in  or  appreciation  of  a  subject 
can  not  be  developed  with  CAI.  j 

-.63 

2. 

63 

.83 

22 

Computers  are  too  impersonal  to  replace  conventional 
instruction. 

-.55 

3. 

30 

1 

.00 

24 

CAI  is  a  potential  threat  to  the  jobs  of  faculty 
members. 

-.31 

2. 

52 

1 

.05 

Factor 

II:     Familiarity  with  Computers 

18 

I  have  become  familiar  with  computers  through  my 

previous  experience.                                                  \  t 

.91 

2. 

88 

1 

.40 

19  < 

I  feel  comfortable  working  with  computers, 
i 

.91 

2. 

86 

1 

.24 

Table  8.7 
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instructors  of  routine  duties  (item  25),  and  student  responsibility  for 
their  own  learning  under  CAI  (item^21) .    For  each  of  the 'other  eight  items 
under  Factor  I  the  modal  response  was  "not  sure"  (see  Appendix  T) .  And 
over  50%  of  the  faculty  responded  "not  sure"  to  the  two  statements  about 
the  general  significance  of  CAI   (items  20  and  29). 

.Factor  II  has  been  termed  Familiarity  with  Computers  since  the 
two  statements  under  this  factor  deal  with  experience  and  work  with- 
computers  (items  18  and  19).     Faculty  responses  to  these  two  statements 
revealed  much  of  the  reason  for  their  uncertainty  about  CAI.     Better  than 
40%  of  the  faculty  disavowed  familiarity  or  comfort  when  confronted  by  a 
computer.     Add  to  these  respondents  anyone  who  chose  "not  sure"  and  we  find 
the  majority  of  faculty  members  probably  lacked  a  background  of  experience 
with  computers.     This  lack  of  computer  experience  perhaps  explains  why  such 
diverse  items  about  CAI  loaded  under  one  general  factor:     faculty  had  no 
practical  basis  for  differentiating  ^mong  the  statements  which  appeared 
under  Factor  I  (see  Table  8.7).     The  frequency  of  neutral  responses  to 
statements  about  CAI  seems  appropriate  and  fair  given  faculty's  prior 
exposure  to  computers • 

Table  8.8  presents  descriptive  statistics  for  statements  in  the  second 
section  of  the  survey.     Although  there  was  little  difference  in  attitudes 
toward  CAI  across  divisions,  there  was  a  pronounced  difference  between 
instructors  and  administrators.     Administrators  tended  to  look  upon  CAI 
more  favorably  than  did  instructors.     Again,  the  responses  to  statements 
included  under  Factor  II  may  provide  a  plausible  explanation  for  the 
results  under  Factor  I.     Administrators'  greater  confidence  and  hope  in 
CAI  might  be  attributed  to  their  greater  familiarity  with  computers. 


Faculty  Attitudes  toward  CAI 
Responses  across  Divisions  and  Positions 
Alexandria  and  Phoenix 
Spring  1974 


Item 
Number 


Respondents 
(N  s  239) 

Mean  sd 


Natural 
Sciences 
(N  =  38) 

Mean  sd 


Social 
Sciences 
(N  ^  24) 

Mean  sd 


Humanities 
(N  2  58) 

Mean  sd 


Applied 
Studies 
(N  a  66) 

Mean  sd 


Instructors 
(N  a  170) 

Mean  sd 


Adminis- 
trators 
(N  *  29) 

Mean  sd 


Factor  I: 
Attitudes 
toward  CAI 


27 
30 
26 
29 
23 
21 
20 
25 
28 
22 
24 


3.57 
3.59 
3.89 
2.47 
3.41 
3.59 
3.14 
3.71 
2.63 
3.32 
2.53 


.79 
.82 
.74 
.77 
.88 
.80 
.90 
.91 
.82 
1.01 
1.05 


3.59 
3.43 
4.00 
2.51 
3.32 
3.73 
3.00 
3.51 
2.73 
3. ,41 
2.95 


.72 
.72 
.66 
.68 
.87 
.76 
■  .70 
1.03 
.76 
.88 
.90 


3.43 
3.39 
3.87 
2.70 
3.59 
3.58 
3.00 
3.58 
2.65 
3.28 
2.75 


1. 


3.34 
3.39 
3.65 
2.70 
3.33 
3.35 
2.98 
3.56 
2.77 
3.55 
2.68 


.79 
.79 
.83 
.71 
.94 
.78' 
.87 
1.01 
.76 
1.00 
1.00 


3.58 
3.68 
3.89 
2,35 
3.31 
3.55 
3.29 
3.82 
2.63 
3.31 
2.18 


.78 
.73 
.66 
.79 
■  .89 
.86 
.99 
.80 
'.78 
1.04 
1.01 


3.46 
3.51 
3.83 
2.57 
3.34 
3.51 
3.05 
3.68 
2.72 
3.42 
2.61 


.78 
.81 
.75 
.75 
.88 
.81 
.92 
.89 
.77 
1.00 
1.04 


4.07 

4.07 

4.28 

1.93 

3.66 

3.93 

3.69 

4.00 

2.14 

2.7,6. 

2.24 


'.64 
.58 
.58 
.64 
.88 
.52 
.79 
.87 
.78 
.90 

1.10 


Factor  II: 
Familiarity 
with  Computers 


Priorities  for 
Success 


18 
19 

31(a) 
(b) 
(c) 
(d) 
(e) 


2.86  1.40 
2.83  1.24 


3.14  14.0 
2.97  1.28 


2.88  1.45 
2.72  1.25 


3.49 
3.59 
2.64 
4.09 
1.38 


1.06 
1.18 
1.08 
1.21 
.83 


3.58 
3.64 
2.72 
3.69 
1.47 


1.16 
1.25 
1.04 
1.29 
1.01 


3.32 
3.68 
2.48 
4.16 
1.64 


1.09 
1.09 
1.14 
1.25 
.97 


2.50  1.30 
2.47  1.13 


2.83  1.44 
2.97  1.30 


2.75  1.40 
2.72  1.24 


3.59 
3.55 
2.50 
4.32 
1.30 


.96 
1.13 
1.07 
1.04 

.73 


3.58 
3.37 
2.78 
4.02 
1.49 


1.12 
1.27 
1.09 
1.26 
.90 


3.47 
3.57 
2.66 
4.11 
1.44 


1.07 
1.23 
1.09 
1.22 
.87 


3.38 
3.62 

3.52 
3.93 
2.55 
3.86 
1.34 


1.32 
.89 

.97 
1.01 
1.10 
1,22 


Table  8.8 
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The  last  question  on  the  faculty* attitude  survey  .(item  il)  had  asked' 

respondents  to  order  several  considerations  forejudging  the  success  of  CAI  / 

t         '  j 

according  to  their  own  priorities.    'They  ranked  five  alternatives  from 
highest  (coded  1)  to  lowest  (coded  5).     In  order,  faculty  priorities 
were:     student  achievements;  student  attitudes;  faculty  acceptance; 
technical  capabilities  and  reliability  of  the  computer  system;  and  finally, 
cost.     These  priorities  for  judging  the  success  of  &kl  certainly  appear 
consistent  with  the  student  focus  maintained  by  community  college  faculty. 

8.2.b    Reactions  to  the  TICCIT  Program 

Another  survey  of  faculty  attitudes  was  conducted  in  the  spring 
term  of  the  1975-76  academic  year.     This  was  nearly  two  years  after  the 
installation  of  TICCIT  systems  in  the  community  colleges  and  toward  the 
conclusion  of  the  first  year  of  the  demonstration  period.     The  TICCIT 
program  had  supported  math  courses  for  four  semesters  at  Phoenix  and  for 
five  quarters  at  Alexandria.     English  courses  had  been  offered  on  the 
TICCIT  system  for  almost  one  fulljyear.    Moreover,  the  colleges  had  incor- 
porated the  TICCIT  program  into  their  curriculum  and  faculty  had  a  definite 
basis  for  responding  to  questions  about  the  program.     The  purpose  of  this 
third  faculty  attitude  survey  was  to  document  instructors  and  administrators' 
reactions  to  the  TICCIT  program. 

I 

Instrument .     The  survey  itself  had  a  question  about  faculty  exposure  . 
to  the  TICCIT  program,  two  questions  about  general  familiarity  with  computers 
33  questions  posed  as  statements  about  the  TICCIT  program  accompanied  by  a 
response 'scale  for  indicating  extent  of  agreement,  and  a  final  question  on 
faculty's  perceptions  of  TICCIT' s  impact  (see  Appendix  I,  Form  003).     The r 
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first  question  on  exposure  to  the  TICCIT  program  established  the  respondent's 
sources  of  inf  ormat  ion^about  the  TICCIT  program.     It  was  a  list  ot  activities 
by  which  faculty  might  have  learned  about  the  program,  such  as^t^aching  a 
TICCIT  section  of  a  course  or  simply  reading -memos  about  the     project  (item 

1,  parts  a-i)  .     Respondents  checked  those  activities  that  formed  their 

*  ■  *  J  ^ 

basis  for  reactions  to  the  TICCIT  program.     The  next  two  questions  (items  2 

and  3)  dealt-with  faculty  exposure  to  computers  in  general.     These  questions 

were  retained  from  the  earlier  survey  of  initial  attitudes  toward  computer- 

assisted  instruction  and  represented  the  factor  already  labeled  Familiarity 

with  Computers  (see  Table  8.7,  Factor  II). 

The  statements  about  the  TICCIT  program  were  either  adapted  from 

earlier  surveys  or  suggested  by  participants'  comments.     Statements  taken 

from  the     previous  faculty  survey  had  the  general  phrase  "computer-assisted 

instruction"  replaced  by  a  specif ic^reference  to  the  TICCIT  pr^ram  in 

order  to  elicit  opionions  about  this  particular  program.     For  example,  "CAI 

relieves  instructors  of  routine  duties"  became  "TICCIT  relieves  instructors 

of  routine  duties."     Comments  made  by  faculty  and  students  prompted  the  - 

development  of  additional  items.     For  example,  faculty  often  cited  the 

self-paced  nature  of  the  TICCIT  program  as  one  of  its  primary  advantages 

while  students  at  times  complained  that  instruction  available  only  on  the 

TICCIT  system  was  less  convenient  for  them  than  the  traditional  combina- 

V 

tion  of  classes  and  homework  exercises.     Both  statements  c^an  be  found 
in  the  third  faculty  attitude  survey  (items  9  and  A).     The  broad  topics 
covered  in  the  survey  might  be  represented  as  student  use  ^^^he  TICCIT 
system  (items  4-12,   31  and  32),  instructors'   tasks  in  relation  to  the 
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TICCIT  program  (items  13-19),  the  TICCIT  curriculum  and  its  delivery  (items 
22-30),  and  global  judgments  of  the  program  (items  20,  21,  33-36). 

„   As,  with  its  predecessor,  the  survey  presented  questions  in  the 
form  o^  statements  followed  by  a  f ive^point  ^response  scale.  Responses 
could  /range  from  strong  agreement ' to  strong  disagreement  with  the^ statement, 
coded  from  fiVe  points  to  one  point,  respectively.     The  scale's  mid-point  t 
was  a  "not  sure11  response  scored  as  three  points.    These  point  assignment 
subsequently  determine^  the  means  and'  standard  deviations  reported  below*' 
for  -each  item*  on  the  survey.     So  as  to  discourage  the  mere  repetition  of 
the  same  response  acrosS  items,  there  were  two  statements  phrased  in  a 
^negative  manner  for  every  three  positive  statements  about  the  program 
(e.g.,  "The  TICCIT  curriculum  is  dull  and  fails  to"  excite  students  about 
the  subject  matter"  and     "TICCIT  promotes  self-confidence  and  a  sense  of 
accomplishment  among  students") . 

The  final  question  the  survey  (item  37)  also  had  a  five-point 
response  scale  but   it  asked  for  faculty  members'   own  evaluation  of  the 
impact  of  the, TICCIT  program.     It  sought  their  assessment  of  TICCIT's 

r 

I  impact  on  student  achievement,  student  attitudes  toward  the  subject  matter*, 
course  completion  rates,  the  quality  of  student-instructor  interaction, 
and  the  quality  of   interactions  among  students.     Faculty  indicated  to  what 
extent  they  thought  the  TICCIT  program  had  affected  these  aspects  of  the. 
educational  process.  / 

Response  Rate.     The  procedure  was  distributing  the  survey  was  the  same 
as  that  for  the  survey  of  beliefs  about  educational"    practices  and  predisposi 
tions  toward  computer-assisted  instruction*.     A  cover  letter,>the  survey,  a 


;,     \  ...  -  ' ;    .    ..  - 

reply ^envelope  and  a  separate  postcard^  oh  th$  status  of  each  form  sent  ^ 

to  all  staff  members  at  the  community  colleges.     After  one  month  another 
letter  w^nt  out  to  those  facultfy  members  Who  had    nbt  yet  responded.  A 
total  of  204  completed  foYms  had  been  returned  within  eight  weeks  of  tt^e 


survey's  distribution  dat<|.    This  represented  a  response  rate  of  52%  s4nce 

391  stafr^  members  had  receive**!  forms  for  completion .  V  /  , 

*  /  ■  -;  <    .  ^ 

This  response  rate  falls  midway  between  the  percentages  of  forms  % 

0,  .    '•■  AY 

returned  in^  the  first  and  second  f  acultj^tt'itude^urveys .     It  is  siibstan- 
tially  higher  th^ir'the  one-third  respon^f  rate/^or  the  first  survey  (Alderman 
&  Mahler,  1977)  due  to  the  brevity -A^^tnyMty  of  this  third  survey.  Yet 
the  rate  is  lower  than  the  <&&t$t  M^  in  the  second 


mrvey.     Since  faculty  ec¥^'$h::/bQ.&  abqu 


t  half  of  the  forms 


sent  to  them,,  thefce  was°  ri^^  site.  Instead, 

we  believe  the  response  tdte- cap*  be  attributed  to  the  need  for  reactions  to 
the  TICCIT  program.     Those  faculty  members  unfamiliar  with  the  program 
simply  did  not  ^respond.    Re##onses  came  from  faculty  who  had  contact  wifth 
the  program  ^nd  thus  had  a  basis  for  reacting  to  statements  about  it. 
Nearly  every  member  of  the  mathematics  and  English  departments  at  the 
colleges  submitted  replies  to  the  survey.  :  It  seems  probable,  therefore^ 
that  the  rjsults  depict  faculty  reactions  fairly. 

Expog&ir£  to  the  TICCIT  Program.     The  activities t checked  by  respondents 
as  contributing  to  their  knowledge  of  the"  TICCIT  project  attest  to  their 
familiarity  with  the  program.    Nearly  one-seventh  of  the  total  number  of 

v  V, 

t espo'ndents  .had  themselvesyiLorkeld  at  a  TICCIT  terminal.     The  majority  of 
those  faculty^ members  responding;  ,to  the  survey  had  attended  an  orientation 
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session  about  the  project  (55%),  readv  articles  of  memoranda  that  dealt  with 
the  program  (57%),  observed  the  TICCIT  program^Ln  operation  (58%),  spoken 

with  students  enrolled  in  a  TICCIT  class  (67%),  and  discussed  the  project 

(  - 

\  - 
Vwlth  their  colleagues  (77%). 

Few  faculty  members  had  ^direct  experience  with  the  TICCIT  program. 

The  program  ^itgelf  precluded  widespread  faculty  participation  by  its  focus 

on  mathematics  and  English  courses.     Only  instructors  in  those  departments 

ha£  an  opportunity  for  close,  daily  contact  with  the  program.     And  most  of 

these  instructors  responded  to  the  survey.     Others  relied  heavily  on 

.     conversations  with  students  and  colleagues  in  forming  impressions  of^the 

program.     Thus,  the  attitudes  documented  through  this  third  faculty  survey 

reflect  reactions  based  primarily  on  indirect  exposure  to  the  TICCIT 

program. 

Dimensions  and  Contrasts.     Eliminating  incomplete  forms  from  the  total 
number  of  204  surveys  left  184  responses  as  a  basis  for  factor  analysis. 
An  item  correlation  matrix  f<^£  the  35  statements  abqut  the  TICCIT  program 
was  calculated  from  this  data  base,  and  a  principal  components  analysis 
with  varimax  orthogdoal  rotations  was. performed .     The  analysis  revealed 
five  major  dimens iod^fcor  faculty  attitudes   toward  the  TICCIT  program. 
Together,  the^e  five  factors  accounted  for  52%  of  'the  variance  in  faculty 


responses   to  individual  items.     The  respective  eigenvalues  jwere  11.55, 
1.94,  1  .70/  1.50  ar\d  1.35.     As  had  been  observed  in  the  discussion  of 
faculty  attitudes  toward  computer-assisted  instruction  in  general,  a  single 
factor  seems  to  dominate  faculty  perceptions  and  explain  by  far  the  most 
variance.     The  percentage  of  variance  accounted  for  in  the  varimax  solution 
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for  the  third  survey  is  almost  identical  to  that  accounted  for  in  the 
varimax  solution  for  the  second  survey's  section  on  CAI .     From  a  technical 
standpoint  the  results  of  the  two  analyses,  one  for  attitudes  towara  L" 
computer-assisted  instruction  generally  and  the  other  for  reactions  to  the 

TICCIT  program  specifically,  seem  consistent. 

> 

f 

The  varimax  solution  for  faculty  attitudes  toward  the  TICCIT  program 

( 

appeals  as  Table  8.9.     Nearly  half  of  the  items  attained  their  highest 
loading  under  the  first  factor.     A  steadily  smaller  number  of  items  loaded 
under  each  of  the  other  four  factors.     This  pattern  reiterates  the  relative 
importance  of  the  first  factor  in  describing  faculty  reactions.  Indeed, 
the  structure  of  the  rotated  factor  matrix  suggests  that  other  factors  ,^may 
be  related  to  the  first.     A  cluster  of  items  loaded  to  a  moderate  extent 
under  both  the  first  and  third  factors  (items  11  and  33-36).     This  indicates 
that  faculty  tended  to  view  the  TICCIT  program  as  a  whole.     Such  a  view 
*might  be  expected  from  the  fact  that  most  faculty  members  relied  on  indirect 
exposure  to  the  program  for  their  knowledge  about  it. 

Table  8.10  presents  the  statements  associated  with  each  of  the 

\ 

factors  in  the  varimax  solution.     These  statements  suggested  the  appropriate- 
ness of  the  label  given  to  each  factor:     Global  Judgments;  Management  of 
Instruction;  Program  Limitations;  Teacher  Role;  and  Familiarity  with 
Computers.     It  should  be  understood  that  these  dimensions  reflect  faculty 
perceptions  of  the  TICCIT  program.     Responses  revealed  fhe  opinions  and 
beliefs  of  the  .college  staff  closest  to  the  program,  idftoely  the  instructors 
and  administrators  at  the  demonstration  sites.     These  opinions  and  beliefs 
represent  fairly  faculty  attitudes  toward  the  project  but  cannot  be  regarded 
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Faculty  Attitudes  toward  the  TICCIT  Program: 
Rotated*  doctor  Matrix 
(after  varimax  Orthogonal  rotation) 
Spring  1976 


Factor  Loadings 

Item 

Number 

I 

II 

III 

IV 

V 

2 

.00 

'  .00 

-.12 

.07 

.80 

3 

-.03 

.00 

.01 

-.10 

,  .77 

4 

-.23 

.48 

.34 

-.30 

-.04 

5 

.71- 

-.16 

-.15 

.03 

.17 

6" 

.66 

^•.15 

-.02  • 

.04 

-.11 

7 

.57 

-.22 

.12 

.20 

.16 

8 

-.24 

.63 

.16 

-.06 

-.04 

9 

.63 

.01 

-.15 

-.13 

-.05 

10 

.60 

-.41 

.03 

.16 

-.01 

11 

.35 

-.35 

.39 

-.21 

.19 

12 

-.20 

.26 

.33 

.16 

-.24 

13 

.04 

.21 

.57 

-.l04 

-.10 

14 

-.20 

.12 

.50 

-.08 

-.07 

15 

.40 

-.48 

-.12 

.36 

.18 

16 

.27 

-.11 

-.03 

.72 

.13 

17 

-.25 

.47 

.37 

-.14 

-.02 

18 

-.30 

.53 

.03 

-.23 

.20 

19 

.04 

.00 

.11 

-.71 

.18 

20 

.50 

-.16 

-.34 

.06 

-.17 

21 

.67 

-.26 

-.34 

.18 

.18 

22 

-.07 

.60 

.21 

-.05 

.06 

23  ,. 

.68 

-.18 

-.22 

.20 

,  «03 

24 

-  .70 

>  .04 

.10 

.01 

-.19 

25 

-.61 

.34 

.38 

.10 

.05 

26 

.66 

-.40 

-.26. 

-.01 

-.05 

27 

.65 

-.28 

-.33 

.00 

-.02 

28 

-.21 

.11 

.56" 

.01 

.21 

29 

.41 

-.18 

-.15 

.12 

-.04 

30 

-.41 

.37 

.24 

.07 

.22 

31 

-.08 

.62 

.10 

.12 

-.10 

32 

.57 

-.47 

-.14 

.23 

.05 

33 

-.56 

.28 

.48 

-.18 

-.12 

34 

.55 

-.22 

-.48 

.27 

.08 

35 

-.45 

.14 

.49 

-.22 

-.25 

36 

.57 

-.16 

-.33 

.33 

.09 

.J  

Table  8.9 


Faculty  Attitudes  toward  the  TLCCIT  Program 
Vnrlm:ix  Primary  Factorn 
Aloxiiiulr  Li  and  Phoenix 
Spring  1976 


Item 
Number 

Factor 

/Stem                      ;                          ._     ■  

Loading 

Mean 

ed 

Factor  1: 

Global  Judgments 

/ 

(N  -  184) 

i 

5/ 

Students  become  active  in  their  own  learning  through  the  use  of  TICCIT. 

.7] 

3.52 

.94 

24 

Students  exercise  control  over  their  instruction  in  the  TITBIT  program. 

.70 

3.62 

i 

.91 

23 

■  I  consider  the  method  of  presentation  on  ^ICCIT  to  be  innovative  and 

.68 

3.37 

.95 

< 

effective. 

21 

I  would  recommend  a  TICCIT  course  to  students. 

.67 

3.24 

1.12 

26 

The  TICCIT  program  develops  student  interest  in  or  appreciation  of  a 
subject. 

.66 

2.95 

.80, 

6 

The  TICCIT  program  tailors  instruction  to  the  individual  silent. 

.66 

3.50 

.94 

27 

TICCIT  promotes  self-confidence  and  a  ;;euse  of  accomplishment  among 
students. 

.65 

3.18 

.85 

9 

A  student  in  a  TICCIT  class  learns  u\    -Asiuttr  own  pace. 

1  97 

.84 

■  25 

The  TICCIT  curriculum  is  dull  and  fails  to  excite  students  about 
the  subject  matter. 

-.61 

2.84 

.92 

10 

Working  on  the  TICCIT  system  improves  students1  learning  strategies 
and  study  habits  in  other  courses. 

i  i  71 

.83 

32 

Students  find  it  easier  t<   Learn  with  TICCIT. 

.57 

2.82 

.76 

36 

TICCIT  ia  one  of  the  most       nif leant  developments  in  education  today. 

.57 

2.96 

1.03 

7 

With  TICCIT,  students  receive  more  individual  attention  from  instructors. 

.57 

2.91 

.93 

33 

The  TICCIT  program  is  not  worth  the  dollars  and  space  which  this 
institution  has  invested  in  it. 

-.56 

3.10 

1.09 

34 

TLCCIT  is  a  valuable  resource  tor  this  Institution. 

.55 

.  3.38 

1.05 

20 

My  colleagues  seem  to  be  favorably  Impressed  by  TICCIT. 

.50 

2.83 

.98 

29 

Students  who've  completed  a  course  on  TICCIT  have  mastered  all  the 
specific  concepts  and  skills  covered  in  the  TICCIT  curriculum. 

.41 

2.93 

.88 

30 

Hie  TICCIT  program  does  not  allow  for  the  integration  and  constant  review 
of  subject  matter  skills. 

-.40 

2.60 

.92 
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Item 
Number 

Stem 

Factor 
"Loading, 

Mean 

,u 

Factor  it: 

Management  of  Instruction 



8 

In  TICCIT  classes  students  had  to  assume  too  much  responsibility  for 
their  own  progress  in  order  to  complete  a  course. 

63 

3.01 

99 

31 

Breakdowns  of  the  computer  system  disrupted  students1  learning  on  TICCIT. 
* 

3.92 

.89 

22 

As  a  pre-packaged  program,  th>%JICCIT , curriculum  affords  instructors 

little  flexibility  in  teaching  a  course. 

V 

.60 

3.22 

.93 

18 

It  is  Afficult  for  an  Instructor  to  manage  students'  learning  in  TICCIT 
classes , 

2.97 

87 

15 

TICCIT  helps  to  make  better  and  fuller  use  of  instructor's  capabilities. 

-.48 

2.84 

1.03 

4 

TICCIT  is  less  convenient  for  students  than  a  ^combination  of  classes  and 
homework. 

.48 

2.91 

1.08 

17 

Instructors  in  the  TICCIT  program  spend  too  much  time  on  mechanical 
problems  and  errors  in  the  curriculum.  6 

.47 

3.33 

.83 

'  Fartor  III 

Program  Limitations 

13 

TlCCIT  is  a  potential  threat  to  the  Jobs  of  faculty  members. 

.57 

2.07 

1  03 

28 

The  TICCIT  program  teaches  only  lower  level  abilities. 

.56 

2.83 

1.06 

14 

I  would  prefer  to  teach  a  regular  section  to  a  TICCIT  section  in  my 
subject  area. 

.50 

3.45 

1.25 

35 

TICCIT  is  a  passing  fad. 

.49 

2.52 

.98 

Factor  IV: 

Teacher  Role 

16 

TICCIT  relieves  Instructors  of  routine  duties. 

.72 

3.16 

1.07 

19 

An  Instructor  has  to  meet  greater  demands  on  his/her  time  in  a  TICCIT 
class  than  in  regular  classes. 

-.71 

2.96 

.73 

Fac  tor  V : 

FAUli  1  t tir  1 1 V  wi  f  h  Prtranitrpru 

2 

I  feel  comfortable  working  with  computers. 

.80 

3.54 

1.10 

3 

1  have  become  familiar  with  computers  from  my  previous  experience. 

.77 

2.98 

1.43 

9 

ERIC 
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as  a  substitute  for  the  data  on  TICCIT' s  Impact  on  student  performance 
or  faculty  duties  presented  elsewhere  in  this  report. 

The  first  factor,  labeled  Global  Judgments,  includes  statements 

about  various  aspects  of  the  TICCIT  program.     Five  items  necessitating  an 

v. 

overall  judgment  of  program  quality  appear  under  this  factor  (items  20,  21, 
33,   34  and  36),     Except  that  most  faculty  "would  recommend  a  TICCIT  course 
to   [their]    students'1  (  item  21),  the  most  frequent  response  to  these  global 
statements  was  "not  sure"   (see  Appendix  T  for  response  dist       utions  by 
Item).     Despite  nearly  two  years  of  experience  with  the  TICCIT  system, 
college  staff  had  yet  to  reach  a  consensus  on  the  value  of  the  program. 
But  they  had  accepted  the  TICCIT  program  as  a  method  for  tailoring  instruc- 
tion to  the  individual  student  and  involving  students  in  their  own  learning. 
Faculty  tended  to  agree  that  students  became  active  in  their  own  learning, 
exercised  control  over  their  instruction,  received  instruction  tailored  to 
the  individual,  and  learned  at  .their  ow     pace  (items  5,   24,   6  and  9)\ 
Since  it  had  earlier  been  found  that  faculty  beliefs  about  educational 
practices  focus  on  the  student,  acceptance  of  the  TICCIT  program  in  terras 
of  the  flexibility  afforded  to  students  may  be  a  positive  outcome. 

Those  statements  listed  under  Factor  II  in  Table  8.10  seem  to  share  an 
emphasis  on  the  management  of   instruction.     Most  of   these  statements  also 
reflect  possible   inconveniences  due  to  the  TICCIT  program.     The  only  clear 
consensus   in  this   factor  was   that  "breakdowns  of   the  computer  system 
disrupted  students'    learning  on  T 1*0 IT"   (Item  22),     And  a  malfunction  in 
any  form  of    instructional    technology  would  be  expected   to  cause  a  disruption. 
Otherwise   the  modal    response*   to   the   items  under  the  second   factor   was  "not 
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sure".    The  second  most  common  response  was  evenly  split  between  those 

t 

indicating  a  positive  reaction  (disagreement  with  items  8,   18  and  4)  and 
those  indicating  a  negative  reaction  (agreement  with  items  22  and  17 
and  disagreement  with  item  15).    Faculty  had  not  formed  a  clear  opinion  of 
the  program's  impact  on  instructors  and  students'  use  of  time.     Indeed,  the 
data  on  student  and  instructor  activities  sometimes  point  to  efficient 
learning  ^or  those  students  capable  of  completing  a  course  on  the  TICCIT 
system  (see  Table  7.7)  and  to  greater  instructor  participation  in  the 
instruction  of  individual  students  (see  Table  8.1),  and  at  other  times  the 
data  indicate  student  hours  expended  with  little  progress  (see  Figure  7.2) 
✓"or  the  burden  of  monitoring  tests  assumed  by  instructors  (see  Table  8.2). 
The  statements  grouped  under  Factor  III  might  together  be  called 
Program  Limitations  since  each  statement  presents  an  item  stem  phrased  in  a 
negative  manner.    Despite  the  direction  of  these  stems,  faculty  responses 
depicted  a  generally  favorable  reaction.    Faculty  did  not  perceive  the 
TICCIT  program  as  a  potential  threat  to  their  jobs  (item  13).  Similarly, 
they  did  not  view  the  program  as  appropriate  for  only  lower  level  abilities 
(item  28)  or  as  just  a  passing  fad  (item  35).     Consistent  with  responses 
given  to  other,  closely  related  items  (e.g.,  items  25  and  27),  it  would 
appear  that  faculty  saw  no     imitations  or  personal  threats  inherent  to  the 
TICCIT  program.     Yet  the  numuer  of  respondents  who  indicated  a  preference 
for  teaching  a  regular  section  of  a  course  over  a  TICCIT  section  was 
twice  as  high  as  the  number  who  preferred  an  assignment  with  the  TICCIT 
program  (item  14).     Considering  the  nature  of  its  statements  and  its 
apparent  correspondence  with  the  first  factor  (see  Table  8.9),  this  third 
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factor  may  be  as  much  an  artifact  of  specific  item  content  as  it  is, a  true 
dimension  of  faculty  attitudes  toward  the  TICCIT  program. 

The  fourth  factor  listed  in  Table  8.10  had  two  statements  load  highly 
under  it.     Both  statements  pertain  to  the  role  of  teachers  under  the  TICCIT 
program.    Almost  the  same  number  of  respondents  were  unsure  aboutMflCCIT's 
impact  on  routine  duties  as  felt  the  program  did  relieve  instructors  of 
such  tasks  (item  16).     By  far  the  largest  number  of  faculty  chose  the  "not 
sure"  response  (57%)  when  asked, whether  "an  instructor  has  to  meet  greater 
demands  on  his/her  time  in  a  TICCIT  class  than  in  regular  classes  (item 
19)".    The  most  frequent  response  to  other  questions  about  teacher  role 
(items  15,   17  and  18)  had  also  been  "not  sure".    Apparently  faculty 
awaited  objective  evidence  on  the  results  of  the  demonstration  and  resisted 
reaching  judgments  based  primarily  on  indirect  exposure  to,  the  TICCIT 
program. 

The  final  factor  given  in  Table  8.10  is  a  repetition  of  a  dimension 
which  had  emerged  in  the  analysis  of  the  earlier  faculty  attitude  survey 
(Table  8.7).    That  dimension  was  labeled  Familiarity  with  Computers  since 
it  dealt  with  faculty's  comfort  in  working  with  computers  and  their  prior 
experience  with  computers.    Again  it  is  perhaps  the  most  4istinct  component 
of  faculty  attitudes.    The  rotated  factor  matrix  shows  just  two  statements 
(items  2  and  3)  that  clearly  belong  under  the  fifth  factor;  these  two 
Statements  had  low  loadings  under  every  other  factor,  and  no  other  statement 
reached  even  a  moderate  loading  under  Factor  V  (see  Table  8.9).    An  equiva- 
lent percentage  of  faculty,  approximately  40%,  had  agreed  and  disagreed 
with  the  statement  "I  have  become  familiar  with  computers  through  my 
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previous  experience  (item  3). 11    Both  the  mean  and  standard  deviation  for 
responses  to  this  statement  (mean  2.98  and  sd  1.43)  came  very  close  to 
those  calculated  for  the  earlier  survey  (mean  2.88  and  sd  1.40).  The 
majority  of  respondents,  however,  reported  that  they  did  feel  comfortable 
working  with    computers:     Over  100  faculty  members  agreed  or  strongly 
agreed  with  item  2  and  fewer  than  40  faculty  disagreed  or  strongly  disagreed 
(see  Appendix  T) .     In  this  respect  the  TICCIT  project  did  affect  opinions. 
The  percentage  of  faculty  uncomfortable  working  with  computers  dropped  to 
half  of  its  former  level.     At  least  the  project  dispelled  apprehension  of 
computers  in  general. 

Table  8.11  presents  descriptive  statistics  for  each  statement  in  the 
third  faculty  attitude  survey  and  does  so  within  academic  divisions  as  well 
as  for  staff  positions.     Unlike  the  survey  conducted  prior  to  the  installa- 
tion of  TICCIT  systems  at  the  community  colleges,  differences  across 
divisions  now  seem  apparent.     Often  one  half  of  a  standard  deviation 
separates  the  mean  response  of  faculty  in  the  natural  sciences  and  humanities 
from  that  of  faculty  in  social  sciences  and  applied  studies  (e.g.,  items 
25,  33,  8  and  15).     And  the  direction  of  the  difference  indicates  a  less 
favorable  reaction  to  the  TICCIT  program  within  the  divisions  of  natural 
sciences  and  humanities.     The  data  on  faculty  attitudes  included  in  the 
summaries  for  the  natural  sciences  and  the  humanities  came  primarily 
from  the  responses  of  instructors  1n  the  math  and  English  departments, 
the  faculty  best  acquainted  with  the  TICCIT  program  (a  summary  of  responses 
from  these  departments  and  for  29  instructors  who  had  taught  TICCIT  sections 
appears  in  Table  8.12).     Indeed,  the  division  of  natural  sciences  should 
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Faculty  Attitudes  toward  the  TICCIT  Program 
Response!  by  Department 

Alexandria  end  Phoenix 
Spring  1976 
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be  well  familiar  with  the  TICCIT  program  by  virtue  of  nearly  two  years  of 
experience  with  the  system  in  math  courses.     This  division  di<i  not  perceive 
the  TICCIT  program  as  one  of  the  most  significant  developments  in  education 
(item  36)  nor  did  the  division  believe  the  project  warranted  the  resources 
which  had  been  devoted  to  it  (item  33).     Judgments  of  TICCIT's  impact  on 
course  completion  (item  37c)  perhaps  explain  these  reactions:     Faculty  from 
the  natural  sciences  knew  that  the  program  had  a  negative  effect  on  course, 
completion  rates  in  mathematics. 

It  has  been  noted  that  administrators  tended  to  view  computer-assisted 
instruction  more  favorably  than  did  instructors  (see  Tabl£  8.8).  Administra- 
tors also  seemed  to  hold  more  positive  opinions  of  the  TICCIT  program  than 
did  instructors,  as  shown  in  Table  8.11.     No  longer  is  greater  familiarity 
with  computers  a  viable  explanation  for  this  difference  £ince  administrators 
and  instructors  appear  to  be  comparable  in  this  dimension  of  their  attitudes. 
Either  the  TICCIT  program  succeeded  in  generating  favorable  reactions  amotlg 
administrators  or  administrators  missed  the  direct  exposure  that  shaped 
instructors'  opinions.     The  contrasts  across  divisions  and  positions 
suggest  that  those  faculty  close  to  the  TICCIT  program  had  a  less  favorable 
reaction  to  it. 

Change .    What  changes  in  faculty  attitudes  occurred  as  a  result  of 
their  exposure  to  the  TICCIT  program?     There  was  little  change  in  their 
opinions  about  student-computer  interactions  or  in  their  global  judgments 
of  computer-assisted  instruction.     Faculty  had  thought  that  CAI  would  make 
students  active  in  their  own  learning  (item  27  on  Form  002,  mean  3.57)  and 
that  it  would  tailor  instruction  to  the  individual  student  (item  23  on  Form 
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02T,  mean  3.40).    The  TICCIT  program  met  their  expectations  ^J>oth  respects 
(items  5  and  6  on  Form  003  with  means  of  3.52  and  3.50,  respectively). 
Similarly,  faculty  did  not  perceive  the  TICCIT  program  as  a  passing 
fad  (item  35  on  Form  003,  mean  2.52)  consistent  with  their  earlier  attitude 
toward  CAI  in  general  (Item  29  on  Form  02T,  mean  2.46).    And  they  were 
unsure  of  whether  or  not  the  TICCIT  program  constituted  a  significant 
development  in  education  (Item  36  on  Form  003,  mean  2.96)  just  as  they  had 
been  unsure  about  CAI  (item  30  on  Form  02T,  mean  2.96). 

There  was  an  obvious  shift  in  faculty  attitudes  toward  the  Impact  of 
computer-assisted  instruction  on  instructors'  responsibility.    Faculty  had 
earlier  tended  to  disagree  with  the  statement  "CAI  is  a  potential  threat  to 
the  jobs  of  faculty  members"  (item  24  on  Form  02T,  mean  2.52)  but  they 
clearly  disagreed  with  the  same  statement  as  it  pertained  to  the  TICCIT 
program  (item  13  on  Form  003,  mean  2.07).    Perhaps  this  further  reduction 
in  the  potential  threat  posed  by  computer-assisted  instruction  can  be 
attributed  to  a  better  grasp  of  the  computer's  strengths  and  limitations. 
Faculty  had  reported  feeling  greater  comfort  in  working  with  computers  . 
fitem  19  on  Form  02T,  mean  2.86,  and  item  2  on  Form  003,  mean  3.54).  And 
part  of  this  improved  understanding  might  be  due  to  a  more  realistic 
concept  of  the  computer's  impact  on  teachers.     Prior  to  the  Implementation 
of  the  TICCIT  program  at  the  colleges,  faculty  generally  believed  that 
computer-assisted  instruction  would  relieve  instructors  of  routine  duties 
and  enable  t^em  to  make  better  use  of  their  capabilities  (items  25  and  30 
on  Form  02T,  leans  3.70  and  3.58,  respectively)  .    After  several  academic 
terms  of  experience  with  the  TICCIT  system,  faculty  became  unsure  of 


whether  the  TICCIT  program  had  affected  instructor  duties  in  such  a  wa 
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(items  16^atia^l5  on  Form  003,  meansf3.16  and  2.84,     respectively).  I 
It  is  difficult  to  capture  the  descriptive  portrayal  of  faculty  ^ 

acceptance  and  teacher  role  in 'a  succinct  and  cle^r  summary i    This -difficulty 

f 

is  perhaps  inherent  to  a  presentation  which  gives  so  much  of  the  context    ■  -I 
necessary  for  appreciating  what  the  treatment  condition  entailed.  The 
first  section  of  this  chapter  was  intended  to  serve  that  purpose.  It 
illustrated  that  instructors  responsible  for  classes  on  the  TICCIT  system 


devoted  substantial  amounts  of  their  time  to  assisting  individual  students  Ko 

J  V  i 

with  their  course  work.     Since  most  of  this  time  was  spent  in  responding  to^J^ 
student  questions  about,  the  subject  matter  and  monitoring  student . attentats  \  j> 

at  mastery  tests,  faculty  did  not  fulfill  the  role  of  manager-ad  vispfc  which 

5r  -. 

the  developers  of  the  TICCIT  program  had  envisioned  for  them.    With  regard 
to  faculty  acceptance,  the  TICCIT  program  had  generally  met  the  expectations^ 
of  college  staff  for  computer-assisted . instruction.     Faculty  did  perceive 
the  program  as  tailoring  instruction  to  the  individual  student  and  engaging 
students  actively  in  their  own  learning^?    However,  those  instructors  w 
closely  associated  with  the  TICCIT  project  often  reacted  to  the  program 
less  favorably  than  their  colleagues.    And  instructors  became  l&ss  certain 
that  computer-assisted  instruction,  particularly  the  TICCIT  progradl, 
benefit  them  in  fulfilling  their  instructional  responsibilities. 
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Chapter  9 

Comments  from  the  Developers  of  TICCIT  ^ 
(The  MITRE  Corporation  and  Hazeltine  Corporation) 

The  addition  of  a  developers1  chapter  to  this  final  report  on  the  1975- 

1976'  TICCIT  demonstration  offers  a  unique  andyhighly  desirable' opportunity  to 

f  \ 
view  educational  research  data  from  more  than  one  perspective.    As  the  developers 

of  TICCIT,  we  wish  to  express  our  appreciation  for  the  opportunity  to  contribute 

our  comments  concerning  the  demonstration  and  its  evaluation. 

Although,  as  might  blPl|fcpected ,  the  developers  and  evaluators  have 

occasionally  differed  concerning  the  interpretation  ancl  weight  of  research 

findings,  the  inclusion  of  this  chapter  is  typical  of  the  high  level  of  coopera- 

tion  between  ETS  and  the  developers  through-out  the  demonstration  period.  This 

♦ 

cooperation  played  a  substantial  role  in  the  development  of  a  successful  TICCIT 

concept;  in  fact,  several  specific  improvements  in  the  TICCIT  system  and  its 

* 

use  were  made  on  the  basis  of  early  communications  with  ETS  evaluators  concern- 
ing the  system1 s  effectiveness. 

This  chapter  has  been  divided  into  two  major  sections.     In  the  first  are 
comments  reflecting  the  developers'  views  of  the  data  collected  by  ETS.  Our 
conclusions  regarding  the  demonstration  are  drawn  not  only  from  the  ETS  data 

but  also  from  studies  conducted  by  the  TICCIT  users  themselves.     The  second 

\ 

section  briefly  describes  the  evolution  of  the  TICCIT  system  since  the 
demonstration. 

Before  discussing  developers1  views,  however,  it  would  be  appropriate  to 
note|that  this  chapter  has  been  itself  the  product, of  extensive  cooperation 
between  several  institutions.     Such  cooperation  was  necessary  because  a  majority 
of  the  TICCIT  system  developers  are  no  longer  employed  at  the  original  developer 
institutions  —  MITRE  Corporation  and  Brigham  Young  University.    Many  of  these 
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individuals  are  now  involved  with  educational  Wnsulting  companies,  including 
WICAT  and  Courseware,  Inc.     The  largest  group  of  developers  to  leave  MITRE 
and  BYU  went  to  Hazeltine  Corporation,  which  now  offers  the  TICCIT  product 
line  oiva  commercial  basis.     The  MITRE  Corporation  has  coordinated  the  develop- 
ment of  this  chapter,  drawing  upon  contributions  not  only  from  various  of  the 
institutions  mentioned  above  but  also  from  the  TICCIT  demonstration  sites. 

9.1    Comments  on  the  TICCIT  Evaluation 

As  developers  of  the  TICCIT  system,  we  have  been  extremely  gratified  by 
the  results  of  the  community  college  demonstration.     The  design,  development, 
"  implementation,  and  demonstration  of  the  TICCIT  system  in  four  years  required 
a  mammoth  undertaking.     The  use  of  TICCIT  instruction  was  a  radical  departure 
from  the  traditional  classroom  instructional  approach,  ^and  it  might  be  said  that 
a  comparison  of  such  a  new  method  as  untested  as  TICCIT  against  a  method  as 
well  established  and  tested  as  classroom  instruction  would  be  a  very  difficult 
one.     Indeed,  the  implementation  of  TICCIT  instruction  was  certainly  adversely 
affected  by  faculty  and  student  skepticism  as  well  as  by  the  typical  problems 
of  a  new  system  under  development,  including  serious  delays  in  production  and 
debug  schedules  of  hardware,  software,  and  courseware.    Yet,  in  spite  of 
the  obstacles,  during  the  first  year  and  a  half  of  its  operation,  the  TICCIT 
system  had  an  unquestionably  positive  impact  upon  student  achievement  in  both 
math  and  English.     The  TICCIT  system  worked:     students  learned  and  continue  to 
learn  the  assigned  objectives,  faculty  learned  to  use  the  system  effectively 
in  many  instructional  programs,  and  administrators  have  continued  to  approve 
the  allocation  of  institutional  resources  for  TICCIT  programs. 
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•As  developers,  however,  we  were  disappointed  by  the  low  completion  ^ates. 


In  attempting  to  find  causes  and  solutions  for  the  problem,  several  innovative 
instructional  management  procedures  have  been  implemented  which/ have  significantly 
improved  TICCIT  completion  rates.     In  fact,  in  some  cases,  completion  rates 


The  major  purpose  of  this  comment  is  to  consider  the  ETS  data  as  well  as 
data  collected  by  the  user  colleges  themselves  during  and  since  the  demonstration 
to  gain  a  broader  perspective  of  the  educational  impact  of  the  TICCIT  system. 
The  first  two  sections  discuss  ETS  and  college  data  concerning  student  achieve- 
ment and  completion  rates,  respectively.     The  third  section  discusses  the 
impact  of  the  role  of  the  instructor  on  completion  rates\andj$bstulates  that 
the  predominant  cause  of  the  demonstration's  low  completion  rates  was  the  lack 
of  a  coordinated  management  scheme  rather  than  an  inherent  TICCIT  bias  against 
students  of  lower  eptrance  ability.  J 


9 . 1 . a    Student  Achievement 

Math.     A  number  of  different  types  of  analyses  of  the  math  achievement 
are^  contained  in  the  ETS  report.     The  first  analysis  involved  eleven  compari- 
sons of  TICCIT  math  sections  and  lecture  sections  of  the  same  course  taught 
during  the  same  academic  term.     The  posttest  scores  of  students  beginning  and 
completing  a  TICCIT  course  within  one  term  were  compared  to  the  posttest  scores 
of  lecture  section  students  of  the  same  entrance  ability,  where  entrance  ability 
was  measured  only  by  the  pretest  administered  at  the  beginning  of  the  term. 
In  eight  of  the  eleven  cases,  the  TICCIT  students  scored  higher  on  the  posttest. 
Levels  of  statistical  significance  were  seen  in  only  two  of  the  eleven  cases; 
both  cases  of  significantly  higher  scores  favored  the  TICCIT  students.  Such 
results  are  encouraging  but  certainly  not  overwhelming.  ^ 


have  surpassed  those  in  comparable  traditional  classes. 
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However,  it  must  be  recalled  that  the  TICCIT  courses  were  "self-paced" 
and  that  no  external  schedules  to  complete  within  one  term  were  imposed  u\>on 
the  TICCIT  students.     As  in  many  other  self -paced  environments,  completions 
within  a  single  academic  term  were  low  in  number  among  the  TICCIT  sections. 
To  expand  the  number  of  student  observations  being  considered,  the  second 
ETS  analysis  dropped  the  time  limitations  of  a  single  academic  term.  There 
were  six  different  math  courses  at  the  two  community  colleges  that  were  so 
analyzed.     In  other  words,  the  achievement  of  all  students  who  finished  a. 
given  TICCIT  math  course  during  the  academic  year  was  compered  to  the  achieve- 
ment of  similar  students  finishing  a  lecture  section  of  the  same  course. 
In  that  analysis  there  were  significant  achievement  differences  in  five  of  the 
six  courses.     All  five  favored  TICCIT. 

A  third  analysis  in  the  ETS  report  examined  ability  and  content  subscores 
for  math  posttests  to  discover  particu^  strengths  or  weaknesses  in  TICCIT 
instruction.     As  might  be  expected,  the  number  of  items  for  each  type  of  ability 
or  category  is  small.     ETS  analysis  indicated  that  although  some  of  the  subscores 
did  approach  reliability  for  group  measures,  most  did  not,  making  any foutcomes 
more  descriptive  than  conclusive.     Few  of  the  categories  resulted  in  significant 

effects;  only  two  significant  effects  were  repeated  across  terms,  and  both 

"r 

were  in  favor  of  TICCIT.     Solution'  of  routine  problems  as  an  ability  level 
involved  positive  TICCIT  effects  in  every  case  and  significant  effects  in  two 
courses.     The  other  repeated  significant  positive  effect  involved  the  content 
category  of  equations.     These  specific  strengths  must  be  attributed  to  the  depth 
of  coverage  of  each  topic  in  a  TICCIT  course  and  the  extent  of  practice 
opportunities.     In  general,  in  those  areas  of  the  math  curriculum  considered 
to  require  the  highest  ability  and  to  be  most  advanced,  TICCIT  outcomes 
showed  consistently  positive  effects.  \ 
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Generally,  the  effects  of  topical  tests  follow  the  pattern  found  in 
posttests.     Although  only  one  of  the  results  was  significant  at  acceptable 
levels,  in  four  of  the  cases  the  results  show  a  sizeable  point  difference 
which  represents  a  positive  TICCIT  effect  equivalent  to  ten  percent  of  the 
total  possible  score  on  the  tests.     Another  indication  of  the  positive  impact 
of  TICCIT  upon  achievement  is  the  fact  that  these  results  apply  to  a  wider  range 
of  students  than  those  taking  a  posttest  since  the  topical  test  were  administered 
earlier  in  the  term  to  a  larger  number  of  students. 

English.     Only  cumulative  learning  as  indicated  through  posttests  was 
analyzed  by  ETS  for  achievement  effects  in  English  courses.     The  posttests 
themselves  were  divided  into  two  parts.     An  objective  test  called  for  responses 
to  multiple-choice  questions  about  grammar  and  writing  structure;   the  second  , 
part  of  the  test  required  the  students  to  write  an  essay  onVa  given  topic. 

In  those  English  courses  in  which  TICCIT  was  used  as  it  was  intended  to 
be  used,  as  a  mainline  instructional  source,   there  were  consistent  and 
statistically  significant  differences,   favoring  TICCIT  over  lecture  sections, 
on  the  objective  test  of  writing  skills.     Student  performance  on  essay  tests 
was  also  better  in  TICCIT  sections,  except  in  a  situation  in  which  a  part- 
time  instructor  unfamiliar  with  TICCIT  conducted  the  TICCIT  class. 

On  the  other  hand,  when  TICCIT  was  not  used  as  the  primary  source  of 

instruction,  achievement  effects  were  mixed.     In  those  classes  using  TICCIT 

adjunctively ,  only  students  of  relatively  high  ability  experienced  gains  on 

the  posttests.     Those  TICCIT  students  of  low  entrance  ability  scored  lower  on 
m 

posttests  than  similar  students  in  lecture  sections.     Use  of  the  system  merely 
as  a  supplement  to  classroom  instruction  simply  does  not  yield  the  same  con- 
sistency of  positive  gains  found  with  the  mainline  applications  in  both 
English  and  math. 
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At  BYU,  there  were  also  two  measures  of  achievement.     All  TICCIT  and 
non-TICCIT  students  were  given  a  departmental  objective  test  of  writing  skills 
and  a  university-administered  writing  competence  exam  of  objective  and 
essay  skills. 

There  were  no  significant  differences  in  the  scores  on  the  departmental 
exam.     However,   the  exam  had  not  been  carefully  designed  to  take  into  account 
the  differences  in  the  TICCIT  and  non-TICCIT  course  outlines  as  had  the  ETS 
exams.     The  departmental  exam  had  been  designed  for  the  non-TICCIT  courses. 

For  essay  tests  and  for  overall  pJass/fail  ratio  on  the  university-  ' 
administered  English  composition  exam,  TICCIT  students  did  significantly 
better  than  non-TICCIT  students.     Over  a  three  semester  period,  BYU  tested 
160  TICCIT  students  and  137  non-TICCIT  students  who  had  had  the  same  instructor. 
Approximately  65%  of  the  TICCIT  students  passed  the  exam  compared  to  only  56% 
of  the  non-TICCIT  students,  primarily  due  to  the  fact  that  TICCIT  students 
achieved^hlgher  scores  on  the  essay  component  of  the  exam. 

Summary.     TICCIT  exerts  a  significant  and  positive  impact  upon  student 
achievement.     For  five  out  of  the  six  observed  math  courses  significant 
positive  results  occurred,  often  resulting  in  TICCIT  posttest  scores  at  lea^t 
ten  percent  higher  than  lecture,  corresponding  to  two-thirds  of  a  standard 
deviation.     In  addition  to  overall  higher  achievement  on  posttests,  TICCIT 
promoted  higher  achievement  in  the  ability  of  students  to  solve  routine  problems 
and  was  equal  to  lecture  in  prompting  adequate  performance  in  the  most 
advanced  content  areas  of  the  math  curriculum.     In  four  of  the  six  math 
courses,  students  in  TICCIT  courses  scored  about  ten  percent  higher  oh  topical 
tests  of  immediate  learning  than  similar  lecture  students.     In  English, 
results  for  courses  comparing  TICCIT  as  designed  and  lecture  likewise 
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demonstrated  significant  positive  effects  for  TICCIT ''students,  resulting  in- an 
estimated  difference  of  five  percent  in  scores  on  objective  tests  of  writing 
skills.     Except  where  inexperienced  instructors  affected  results,  TICCIT 
students  also  scored  higher  in  essay  tests. 

9.1.b     Course  Complet ion  Rates 

The  percentage  of  students  completing  TICCIT  courses  within  an 
academic  term  varied  trenemdously  among  the  various  applications  of  the  system. 

The  most  reasonable  explanation  for  the  variations  appears  to  be  the  environment 
in  which  the  TICCIT  student  learns.     In  existing  literature,   it  is  fairly  clear 
that  without  some  type  of  external  management  effort,   fewer  students  will 
complete  an  individually-paced  course  than  an  instructor-paced  course  (Atkins 
&  Lockhart,   1976;   Coldeway  &  Keys,  1976;  Coldeway,  1974;  Lloyd,  McMullin, 
&  Fox,  1974;  Coldeway  &  Scheller,  1974). 

In  the  traditional  classroom  situation,  motivation  to  complete  the 
course  within  a  given  time  frame  is  to  some  degree  built-in,  usually  in  the 
form  of  scheduled  assignments  and  tests.     To  supply  the  motivation  to  complete 


the  course  on  schedule  in  an  otherwise  self -paced  TICCIT  environment,  the 
instructor  must  adopt  the  role  of  manager/advisor,  providing  individual  students 
with  progress  and  achievement  goals  and  evaluating  each  student's  performance 
in  relation  to  those  goals.     Completion  rates  in  TICCIT  courses  will  depend 
upon  the  extent  to  which  an  instructor  adopts  that  role.  ' 

In  the  original  conr  'inity  college       .lementation,   there  was  no  emphasis 
upon  completion  as  a  program  outcome;  however,  early  in  the  demonstration  it 
became  apparent  that  high  completion  rates  would  not  be  achieved  without  a  more 
structured  environment.     Therefore,   the  TICCIT  users  at  BYU  established  the 
use  of  TICCIT  in  environments  which  included  a  weekly  scheduled' class  meeting, 
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small  group  discussions,  individual  student/ teacher  meetings,   scheduled  quizzes, 
or  other  features  designed  to  motivate  the  student  to  complete  within  the 
academic  term. 

The  BYU  model  produced  TICCIT  completion  rates  generally  higher  than  in 


instruction  was  65%.     Classes  taught  by  one  of  the  TICCIT  math  authors  averaged 
higher  rates  than  classroom  rates.     Two  classes  taught  by  teachers  previously 
unfamiliar  with  TICCIT  were  mixed;  one  had  a  completion  rate  of  72%,   the  other 
33%.     Rates  for  TICCIT  English  classes  were  very  favorable,  ranging  from  88% 
to  96%.     Those  rates  exceeded  the  very  liberally-calculated  completion  rate  of 
82%  for  the  classroom  (the  lecture  sections  did  not  include  in  the  rate 
cal     ^tion  any  of  a  large  number  of  students  withdrawing  before  the  third 
WL  V     f  class,  although  the  TICCIT  sections  did).     Clearly,  when  efforts  are 
expended  to  obtain  high  completion  rates,  it  is  possible  to  do  so  in  an 
environment  which  attempts  to  pace  students  within  certain  time  periods. 

Although  the  community  college  implementations  of  TICCIT  did  not  generally 
include  such  concerted  efforts  to  increase  completion  rates,  the  TICCIT 
English  instructors  were  generally  very  imaginative  in  their  approaches  to 
directing  the  TICCIT  classes,  using  features  such  as  class  meetings  and  group 
discussions.     It  was  apparent  that  those  efforts  did  result  in  completion 
rates  that,  although  lower  than  lecture  classes  at  the  community  college,  were 
significantly  higher  than  those  in  math  classes  in  which  there  was  basically 
no  effort  to  pace  student  learning  according  to  any  schedule.     In  three  of  the 


five  cases  in  English  there  were  significant  nega  impacts  upon  completion 

attributable  to  the  TICCIT  condition,  but  in  every  case  the  English  completion 
rates  were- substantially  higher  than  found  in  the  TTCCTT  math  sections.  The 


comparable  lecture  sections. 


In  math,   the  average  completion  rate  for  classroom 
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rates  for  English  were  generally  10%  to  20%  lower  than  classroom  completion 
rates,  which  ranged  from  60%  to  75%. 

The  math  courses  at  both  community  colleges  were  generally  implemented 
without  any  of  the  tools  discussed  above  that  would  tend  to  motivate  pacing 
within  the  academic  term.     Total  self-pacing  was  the  rule  for  math  classes; 
there  was  no  penalty  for  re-enrolling  for  the  same  course  in  subsequent  terms. 
There  appeared  to  be  little  or  no  discussion  _>^th  individual  students  concerning 
learning  pace,  and  there  were  no  group  or  class  discussions.     The  results  of 
the  analysis  do  show  significant  treatment  effects  upon  completion.     In  10 
of  the  12  math  classes  analyzed,  significant   (at  p  <  .001  levels  in  all  but 
one  case)  and  negative  treatment  effects  were  associated  with  the  TICCIT 
sections;   the  two  remaining  sections  showed  no  significant  treatment  effects. 
The  rates  for  TICCIT  sections  were  generally  about  half  the  size  of  lecture 
rates,  which  were  about  40%  to  55%. 

Implementations  of  more  stringent  management  efforts  at  the  community 
colleges  since  the  demonstration  have  resulted  in  substantial  increases 
in  the  TICCIT  completion  rates.     The  latest  figure  from  pne  community  college 
shows  a  completion  rate  in  TICCIT  English  (72%)  which  surpassed  the  rate  in 
the  lecture  sections.     The  other     ollege  has  reported  completion  rates  in  math 
which  indicate  that  when  allowing  students  three  quarters  to,  complete  a 
course,  slightly  more  TICCIT  students  (48%)  than  lecture  students  (44%) 
complete  the  course. 

9.1. c    Comments  on  Completion  Rate  Data 

Those  who  continue  to  develop  and  market  the  TICCIT  system  have 
been  very  pleased  with  TICCIT  evaluation  results.     The  achievement  data 
consistently  demonstrate  significant  and  substantial  positive  gains  for  TICCIT 
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students  in  both  math  and  English.     The  measure  of  success  for  TICCIT  was  to 
be  its  impact  upon  achievement  compared  to  traditional  instructional  methods. 
In  that  light,   it  must  be  said  that  even  in  its  initial  operation,  TICCIT 
was  highly  successful ,   teaching  math  and  English  students  often  better  than 
was  done  in  traditional  classes. 

Perhaps  the  most  gratifying  results  were  found  in  the  English  composition 
courses.     Efforts  to  develop  computer-assisted  materials  for  composition 
instruction  were  met  with  constant  skepticism.     Yet,   the  evaluations  have 
shown  that  completion  rates  in  TICCTT  English  classes  have  been  similar  to  or 
surpassed  completion  rates  in  lecture  classes.     More  important  was  the  finding 
that  those  students  completing  TICCIT  scored  consistently  higher  on  objective 
and  essay  posttests. 

Similarly,   students  in  TICCIT  math  classes  performed  consistently  better 
on  course  posttests  and  on  topical  tests  than  their  counterparts  in  lecture  s 
sections.     Although  completion  rates  for  the  highly  managed  environment 
did  surpass  lecture  rates,  the  rates  for  TICCIT  math  classes  within  the 
evaluation  context  were  disappointingly  low.     Completion  was  not  an  outcome 
highly  emphasized  during  the  demonstration,  but  the  higher  completion  rates 
'found  in  the  BYU  study  would  have  been  just  as  desirable  in  the  community 
college  setting. 

A  very  important  question  for  those  involved  in  current  development  of 
TICCIT  and  CAI  in  general  is  why  completion  rates  in  math  were  so  much  lower 
at  the  community  colleges  than  the  English  rates,   especially  since  the  two 
courses  were  basically  identical  in  design/    It  has  already  been  suggested 
that  the  difference  lies  chiefly  in  the  degree  of  instructor  involvement  in 
managing  student  progress. 
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The  explanation  for  generally  lower    TICCIT  completion  rates  apparently 
favored  by  ETS  was  that  the  TICCIT  system  itself  may  favor  students  of  high 
ability  to  the  detriment  of  low  ability  students.    While  the  completion  rate 


relate  to  completion  within  a  sir.     e        :n      Such  results  relate  more  to  the 
speed  of  acquiring  jnastery  of  the  ouj <•<  ^ives  rather  than  to  the  ability  to 
master  the  objective  at  all.     This  consideration  and  others  which  are  discussed 
below  tend  to  dispel  the  notion  that  TICCIT  is  suitable  only  to  "high  ability11 
students . 

The  ETS  conclusion  concerning  ability  is  based  upon  the  observation  of 
differences  in  pretest  means  of  students  finishing  the  course  in  TICCIT 
sections  and  those  students  ^finishing  the  lecture  sections.     The  differences 


differences  in  pretest  mean  scores  became  much  smaller,  primarily  because  of 
the  addition  in  later  terms  of  the  scores  of  students  taking  more  than  one  term 
to  complete.     The  effect  of  this  can  be  seen  in  the  analysis  of  the  fall  term 
of  beginning  algebra  at  Phoenix.     The  pretest  mean  for  students  finishing 
TICCIT  was  44.86  while  for  lecture  it  was  41.28.     Logically,  one  might  conclude 
that  any  difference  in  posttest  scores  (actual  scores  for  TICCIT  and  lecture 
were  40.06  and  38.82,  respectively)  must  be  due  to  the  high  entrance  ability. 
However,   further  analy.11  is  limiting  the  comparison  to  higher  ability  students 
(defined  in  this  case  by  pretest  scores  greater  than  or  equal  to  35)  still 
showed  positive  achievement  effects.     Later  analysis  eliminating  the  constraint 
of  completions  within  a  term  included  a  sizeable  group  of  students  who  had 
completed  TICCIT  math  during  the  spring  semester.     The  mean  pretest  score  of 
this  group  was  33.92,  which  is  below  the  earlier  defined  high  ability  level 


figures  for  the  math  courses  do  ar 


ated  to  entrance  ability,  the  figures 


in  the  fall  terms  were  particularly  large.  However, 
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of  35,  below  the  pretest  mean  of  lecture  students  completing  the  fall  semester 
"(38.64),  below  even  the  mean  of  the  lecture  students  from  the  fall  who  did 
not  finish  within  one  term  but  also  completed  in  the  spring  (34.62),  and  below 
the  mean  pretest  scores  of  all  students  tatting  the  pretest.     Yet  this  broadening 
of  the  applicable  entrance  ability  did  not  subtract  from  the  overall  achievement 
results.     In  fact,  as  mentioned  earlier,  the  inclusion  of  all  the  students 
requiring  more  than  one  term  to  complete  increased  the  impact  of  achievement 
results.     Although  the  view  of  the  TICCIT  completion  and  achievement  results 
within  a  single  academic  term  appears  to  support  an  argument  for  TICCIT 
suitability  only  to  high  ability  students,  the  results  across  terms  weakens 
the  argument  with  the  inclusion  of  substantial  numbers  of  students  with  much 
lower  entrance  abilities  who,  nonetheless,  add  a  greater  degree  of  significance 
to  the  positive  achievement  data.     It  does  appear  that  students  with  higher 
pre-test  scores  are  able  to  finish  TICCIT  courses  faster  than  those  with  lower 
scores,  a  finding  not  unusual  with  individually-paced  learning  systems. 

An  additional  factor  which  weakens  the  high  entrance  capability  argument 
is  the  topical  test  analysis.     For  topical  testing,  an  even  broader  segment 
of  the  student  population  was  included.     Unfortunately,  no  detailed  analysis 
is  made  of  students  completing  topical  tests  in  terms  of  entrance  ability. 
The  comments  in  the  evaluation  imply  that  the  group  taking  the  topical  test 
more  inclusive  than  the  group  completing  TICCIT  but  did  not  include  the  entire 
population  beginning  the  course.     The  students  were  characterized  as  those  able 
to  make  progress  towards  completion  on  TICCIT.     Again  with  the  enlarged 
population,  there  are  still  substantial  achievement  gains  for  TICCIT  stunts 
in  five  of  the  six  courses.     Although  the  data  are  not  clear,  the  implication 
from  the  topical  test  results  is  that  the  entrance  ability  argument  must  be 
still  further  weakened.  J 
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*  ,      the  ETS  report  seems  to  imply  that  the  topical  test  results  reinforce 
t 

the  argument  that  TICCIT  is  suitable  for  students  of  higher  ability  because 

•  *  t 

the  TICCIT  population  taking  topical  tests  was  still  smaller  and  more  select 
than  the  population  originally  enrolled  in  the  course.     That  fact  might  be 
interpreted  as  evidence  in  support  of  the  contention  that  TICCIT  is  better 

\ 

suited  to  highly  capable  learners.     But  the  context  of  that  evaluation  is  a 
^comparison  of  TICCIT  and  traditional  instruction;   therefore,  the  statement  that 
TICCIT  would  be  suitable  .otily  for  high  ability  students  implies  that  the  lecture 
•■    condition  has  a  less  negative  impact  than  TICCIT  upon  those  of  lower*  ability. 
It  is  clear  from  ETS  data,  however,  that  the  traditional  instructional  method 
also  exerts  a  strong  bias  against  the  lower  ability  student.     The  issue  then 
is  not  simply  whether  TICCIT  exerts  a  bias  against  low  ability  students  but 
whether  that  bias  is  significantly  greater  than  that  already  existent  in  tra- 
ditional instruction.     Without  substantial  differences  between  any  bias  exerted 
by  TICCIT  and  bias  exerted  by  lecture  methods,   the  statement  that  TICCIT  is 
Suitable  for  high  ability  students  could  not  be  justified.     On  the  basis  of 
the,, Results  of  across-term  analysis  and  topical  testing  discussed  above,  there 
do  not  appear  to  be  such  differences  to  justify  the  conclusion  that  TICCIT 
applications  should  be  limited  to  those  involving  only  highly  capable  learners. 

Further  arguments  against  the  general  hypothesis  that  TICCIT  is  not  suitable 
.for  students  of  less  than  high  ability  are  found  in  the  following  observations. 
First  of  all,  the-math  courses  under  considerat ion  were  fundamental  courses 
in  .community  colleges  with  very  liberal  enrollment  policies.     A  large  number 
of  the  classes  for  which  data  were  taken  did  not  qualify  as  college  credit 
courses  but  were  J,evelopmental  classes  to  compensate  for  pre-college  deficiencies. 
They  are  necessary  and  important  courses,  but  it  may  be  difficult  to  support 
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any  general  statement  concerning  "high  ability'1  students  on  the  basis  of 
observations  only  from  these  courses.     Perhaps  even  more  interesting  is  the 
relationship  between  two  of  the  courses  at  Phoenix.     Generally,  students  enroll 
in  Math  007  only  because  they  have  not  demonstrated  the  entrance  ability,  as 
judged  by  performance  on  a  pretest,  required  for  Math  106.     By  definition, 
all  students  in  Math  007  would  be  of  lower  entrance  ability  (i.e.,  lower 
pretest  scores)  than  those  in  106.     Yet  the  analysis  for  Math  007  demonstrated 
one  of  the  strongest  positive  effects  for  TICCIT  students. 

In  short,   there  appears  to  be  satisfying  evidence  that  TICCIT  can  be 
successfully  applied  in  learning  environments  in  which  there  are  varied  entry 
abilities.     Although  entrance  ability  must  be  considered  when  implementing 
TICCIT,   the  consideration  should  not  preclude  those^ of  low  entry  levels. 
TICCIT  was  built  on  a  mastery  hypothesis;  that  is,  given  adequate  instruction, 
defined  as  having  a  complete  learning  hierarchy,  properly  stated  rules  with 
alternate  versions  of  the  rule,  and  extensive  files  of  examples  and  practice 
problems  with  helps,  any  student  with  the  prerequisites  for  a  given  lesson 
could,  given  enough  time,  pass  that  lesson.     The  TICCIT  math  course  assumed 
a  minimum  set  of  prerequisites  for  entering  students.     The  effect  of  not 
having  the  prerequisites  was  more  pronounced  in  math  than  in  English  since  math 
is  a  cumulative  subject.     Large  numbers  of  the  community  college  students  did 
not  possess  the  minimum  prerequisites  required  by  TICCIT,  yet  they  were 
allowed  into  the  TICCIT  courses  as  they  were  allowed  into  the  lecture  courses. 
The  design  philosophy  of  TICCIT  does  not  specify  that  all  students  who  lack 
the  prerequisites  would  be  able  to  succeed  —  no  matter  how  much  practice  on 
a  higher  order  math  skill  they  were  able  to  engage  in.     Rather,  the  TICCIT 
design  hypothesis  is  that  for  such  students,  other  TICCIT  lessons  could  be 
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developed  which  would  give  them  the  prerequisites  needed  to  enable  them  to 
complete  the  higher  lessons  in  a  manner  most  efficient  for  their  particular 
learning  rate  and  aptitudes. 

Evidence  that  the  students  did  lack  the  necessary  prerequisites  is  found 
in  the  large  numbers  of  students  who  were  not  able  to  complete  the  first 
TICCIT  unit  or  who  took  an  extremely  long  period  of  time  to  do  so.     Once  students 
were  able  to  complete  the  first  unit  in  math,  chances  for  their  successful 
completion  of  the  course  were  gvesfc^y  improved.     However,  it  cannot  be  said 
that  the  TICCIT  system  itself  is  inadequate  if  students  not  possessing  the  ^ 
presumed  ^prerequisites  have  difficulty  achieving  mastery  on  higher  order  lessons. 
If  the  ETS  data  are  viewed  as  formative  evaluation  data,  it  is  clear  that  the  * 
hierarchy  must  be  extended  downward  and  that  existing  material  should  be 
simplified.     That  the  model  is  less  effective  for  less  able  students  does  not 
seem  to  be  a  valid  conclusion. 

The  entrance  ability  hypothesis  does  not  explain  adequately  the  low 
completion  rates  in  the  community  college  math  courses.     Comparing  the 
completion  rates  of  math  to  those  of  the  self-paced  programmed  instruction 
makes  it  reasonable  to  assume  that  the  problem  of  low  completion  rates  is  a 
generic  problem  with  self-managed  instruction.     The  rates  found  in  TICCIT 
were  comparable  to  the  rates  in  the  self-paced  programmed  instruction  at 
Alexandria  and  to  the  historical  rates  of  similar  programs  at  Phoenix.  The 
evaluation  report  mentions  the  problem  a  number  of  times,  referring,  for 
example,   to  the  fact  that  many  students  failed  to  attend  classes  regularly, 


or  at  all,  when  pressures  from  other  non-self-paced  courses  increased. 
Records  of  the  number  of  hour;;  spent  on-line  also  indicated  that  students  not 
completing  the  course  spent  fewer  hours  on  the  system  than  the  number  of  hours 
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assigned  for  classroom  usage  in  the  lecture  sections.     (For  example,  a  study 
conducted  by  Alexandria  indicated  that  average  usage  for  students  not  completing 
TICCIT  sections  was  approximately  30  hours.     The  lecture  sections  met  for  50 
hours  per  term.)     There  is  apparently  a  problem  with  the  management  of  student 
progress  which  greatly  contributes  to  lower  completion  rates. 

The  management  problem  is  not  surprising,  however.     From  the  beginning 
of  the  project,  TICCIT  developers  Understood  that  such  problems  existed  with 
self-paced  instruction  and  did  attempt  to  create  solutions  in  advance.  The 
TICCIT  math  and  English  courses  were  both  designed  to  be  implemented  in  an 
environment  in  which  the  instructor  took  an  active  role  in  the  management  of 
student  progress   (Black,  1974;  Bunderson,  1973).     The  originally  defined  role 
of  manager-advisor  to  be  assumed  by  instructors  included  goal-setting  and 
performance  evaluation  behaviors.     Such  a  model  was  implemented  much  more  fully 
in  the  English  than  in  the  math  courses.     As  has  been  seen,  completion  rates 
in  English  were  higher;   in  fact,   the  lowest  completion  rate  in  any  English 
course  was  higher  than  that  for  any  math  section. 

The  English  and  math  course  designs  themselves  were  simply  not  divergent 
enough  to  result  in  such  large  completion  differences.     However,  a  review 
of  the  implementation  contexts  shows  striking  differences.    . ETS  observed 
that  English  instructors  took  an  active  role  in  prescribing  study  materials, 
conducting  small  group  discussions  about  common  difficulties,  and  handling 
individual  student  problems.     On  the  other  hand,  math  students  had  much  less 
instructor  contact,   and  tlfe  great  majority  of  contact  that  did  occur  was  centered 
on  content  problems  rather  than  individual  progress  problems.     At  Phoenix, 
students  were  not  even  assigned  to  particular  math  instructors;  consequently, 
the  instructors  assigned  to  the  lab  in  a  given  hour  may  not  have  been  able 
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even  to  identify  individual  students.     In  short,   there  appear  to  have  been 
substantial  differences  between  both  the  quantity  and  quality  of  instructor 
interaction  with  math  and  English  students. 

It  is  perhaps  understandable  that  better  management  efforts  were  not 
implemented.     The  developers  did  attempt  to  provide  features  in  the  system, 
such  as  various  reports  concerning  individual  student  progress,  but  for  various 
reasons  no  systematic  method  of  handling  the  management  problem  was  provided. 
It  is  to  the  credit  of  the  colleges  themselves  that  programs  were  developed 
in  the  English  courses  that  did  improve  student  completion  rates  through 
better  management.     The  methods  implemented,   including  small  group  discussions 
and  closer  involvement  in  individual  assignments,  could  and  should  also  be 
used  in  mathematics.     The  possible  impact  of  management  was  not  discussed  in 
the  ETS  evaluation  of  math  completion  rates,  possibly  because  none  of  the 
math  sections  dealt  with  the  problem  and  because  no  obvious  solutions  ^jj^^fe^ 
the  problem  were  ever  suggested  or  implemented.     But  because  of  the  basic 
similarities  between  the  courses,   there  is  reason  to  believe  that  similar 
management  efforts  would  also  increase  math  completion  rates.     The  evaluation 
did  indicate  a  strong  correlation  between  time  spent  on-line  and  the  number 
of  math  lessons  completed.     To  improve  the  rate  of  completion,   it  would 
appear  then  that  measures  are  needed  to  improve  student  motivation  to  spend 
an  adequate  amount  of  time  on  the  system. 

Further  evidence  for  the  efficacy  of  management  efforts  is  found  in  the 
BYU  math  implementation,  discussed  earlier  in  the  completion  rates  section, 
in  which  TICCIT  completion  rates  did  exceed  tho^e  of  the  classroom.  The 
literature  would  also  support  the  hypothesis  that  increased  management  efforts 
will  result   in  higher  completion  rates   (Lloyd,   el.   al.,    1974;   Miller,  Waver 
Semb,  1974;  Malott  &  Svinicki,  1969). 
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s,   The  developers  were  highly  pleased  with  the  results  of  the  evaluation. 
Very  clearly  positive  achievement  results  for  instruction  in  English  was  a 
^sult  expected  by  very  few  people.     Yet  the  results  do  show  that  grammar  and 
composition  can  be  taught  systematically  with  positive  effects  on  student 
writing  capabilities.     Even  iiv^^e^t|\ial  year  of  implementation  of  a  radically 
new  Instructional  method-,  neairly^Ss  many  students  finished  the  course  with 
TICCIT  as  with  the  established  instructional  method  and  performed  significantly 
better  on  achievement  tests.     As  instructors  have  become  experienced  with 
the  system,   its  effectiveness  continues  to  be  manifested:     completion  rates 
now  often  exceed  those  of  the  classroom.     Achievement  for  math  students  on 
TICCIT  is  equally  pleasing.     Distracting  slightly  from  that  enthusiasm  were 
the  lower  completion  rates  found  with  the  math  sections  within  the  ETS  eval- 
uation.    However,   the  developers  find  no  reason  to  believe  that  there  is  any 
inherent  characteristic  of  TICCIT  instruction  that  would  prohibit  completion 
in  math  from  being  equal  to  that  found  with  English.     There  is  ample  evidence, 
including  already  improved  completion  rates,   that  the  management  techniques 
used  to  improve  completion  for  English  classes  are  just  as  effective  in 
promoting  completion  rates  in  math  comparable  to  or  exceeding  those  in  lecture 
classes.     TICCIT  is  a  very  powerful  instructional  tool,  one  which  should 
continue  to  have  a  great   impact  on  the  future  of  educational  practices. 

9 . 2     Post  Demonstration  Developments  s- 

Over  the  past   few  years,   the  TICCIT  system  has  continued  to  expand  and 
develop.     Several   significant:  events   in  the  system's  recent  history  include  the 
commercialization  of  the  system  and  the   installation  of  four  new  TICCIT 
systems,   two   for  military  training  and  two   for  special  education.  Application 
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of  TICCIT  technology  to  new  training  situations  and  subject  areas  has  also 
continued  at  the  demonstration  community  colleges  and  Brigham  Young  University. 
Since  Hazeltine  Corporation  acquired  commercial  rights  to  the  TICCIT  system, 
the  TICCIT  concept  has  continued  to  evolve  in  many  directions.     In  conjunction 
with  its  uses  in  many  different  instructional  and  training  situations,  the 
TICCIT  system  will  continue  to  implement  state-of-the-art  technology  to  solve 
the  needs  of  its  users  in  the  most  effective  and  efficient  manner  possible. 

9. 2. a     Commercialization  of  the  TICCtT  System 

One  of  the  major  goals  of  the  developers  of  the  TICCIT  system  was  the 
creation  of  a  commercially  successful  computer-based  instructional  system. 
The  first  step  toward  achieving  a  market  success  was  to  secure  a  commercial 
firm  to  manufacture  and  market  the  TICCIT  system  following  the  initial  demonstra 
tion.     Thirty  companies  formally  evaluated  the  TICCIT  system  as  a  potential 
product  line,   eight  of  which  carried  on  extended  dialogues  with  MITRE  and  NSF. 
Of  the  eight,  MITRE  selected  Hazeltine  Corporation,   partly  because  of  Hazeltine' 
commitment  to  continue  the  development  of  the  system  and   the  support  of"  the 
existing  TICCIT  inst  .    I  at  ions . 

In  late  1.976,   Hazt    '  ine  Corporation  acquired  the  rights  to  the  TICCIT 
system  and  began  comme*      il   marketing  of  the  system. 

9 . 2 . b     New  T1CC I T  Install  a t ions 

-j 

Kven  before  Hazeltine  Corporation  acquired  commercial    rights   to  the 
TICCIT  system,   several    institutions-  were4   interested    in  purchasing  TICCIT 
systems.      lie  fore  the  end  ol    the  T  ICC  IT  demonstration  at   the  community  colleges, 
the  MLTRK  Corporation  oust om-des i gned  and    installed  TICCIT  systems  at  North 
Island  Naval    Air  Station    in  San  Diego,    California;   Cecil    Kield  Naval  Aii 
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Station  in  Jacksonville,  Florida;  at  the  Model  Secondary  School  for  the  Deaf 
(MSSD)  on  the  campus  of  Gallaudet  College  in  Washington,  D.C.;  ari^in  Amherst, 
New  York.     With  these  four  systems,   TICCIT  technology  was  applied  to  two  new 
fields  of  instruction  —  military  training  and  special  education. 

Military  Training.     The  TICCIT  system  is  currently  being  used  for  air 
crew  training  fot  the  S-3A  Viking  antisubmarine  aircraft.     Initial  results 
from  a  program  evaluation  at  North  Lsland  Naval  Air  Station  indicate  that  the 
TICCIT  system  is  a  significantly  more  efficient  training  medium  than  the 
traditional  media.     Through  the  use  of  TICCIT  instruction,   training  time  was-^ 
,  reduced  by  as  much  as  one  half   (Walker,   1978).     The  Navy  is  continuing  its  use 
of  the  TICCIT  system  for  S-3A  training  and  development  of  TICCIT  courseware. 

Special  Education.     The  TICCTT  system  is  currently  being  used  in  two 
very  innovative  special  education  projects. 

In  Amherst,  New  York  a  custom-designed  TICCIT  system  is  used  to  deliver 
computer-based   Instruction,   games,   and  simulations  to  the  homos  of  homebound 
handicapped  children.     The  children  receive   interactive  instructional  displays 
directly  on  their    home  television  sets  and   interact  with  the  displays^ using  a 
special  keyboard  and   telephone   link  to  the  system's  central  computer.  The 
project,   serving   100   families,    is  the  first   to  attempt  such  large-scale  home 
de  livery  o  f    commit  er- based    instruction.  v 

At:   the  Model   Secondary  School    for   the  Deaf,    located  on   the  campus  of 
Callaudet  College    in  Washington,    D.C.,    the  TICCIT  system   is  being  used  to 
deliver  'instruction   to  .md    facilitate  communication  between  deaf  students. 
The  MSSh's  TICCIT  system    is    integrated  with  the  school's  r I osed-c I rcu  1 1  cable 
television   to  provide  a  wide   range  ni"  services.     The  MSSD  student  terminals 
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are  used  interchangeably  for  computer  generated  instructional  displays, 
commercial  and  local  television  broadcasts,  and  terminal-to-terminal  video 
communications. 

9.2.C    TICCIT  Demonstration  Sites  —  An  Update 

Some  of  the  most  convincing  evidence  of  the  TICCIT  system1 s  success 
may  be  gleaned— £*^>m  a  look  at  the  current  status  of  the  system  at  both  of  the 
demonstration  community  colleges  and  Brigham  Young  University.    All  are  actively 
used  for  coursew^/e  development,  the  delivery  of  instruction,  and  research. 
Since  the  demonstration,  all  three  institutions  have  expanded  their  systems 
as  well  as  their  applications  of  TICCIT  instructional  technology. 

Brigham  Young  University.     Since  the  end  of  the  demonstration  period, 
BYU's  TICCIT  system  has  undergone  several  dramatic  changes.     With  the  end 
of  the  demonstration,  BYU's  role  as  the  primary  courseware  development  system 
for  the  community  colleges  ceased.     The  TICCIT  system  had  to  find  a  role  within 
the  university.     After  serveral  moves,  the  system  is  now  located  in  the  Learning 
Center  in  BYU's  new  Harold  B.  Lee  Library  and  has  been  integrated  into  the 
Department  of  Learning  Services  where  it  is  used  for  research,  courseware 
development  and  instruction  in  English,  Math,  French,  Italian,  Reading,  Nursing 
and  English  as  a  Second  Language.     BYU  also  has  plans  to  expand  their  hardware 
system  to  permit  the  storage  of  even  more  courseware .through  the  addition  of 
two  twenty-five  megabyte  disk  drives. 

Northern  Virginia  Community  College.     Since  the  demonstration  period, 
the  TICCIT  system  has  been  fully  integrated  into  the  campus  Learning  Resd^rce 
Center  where  it  continues  to  be  used  to  teach  both  English  and  Math.  Toward 
the  end  of  the  demonstration  period,  NVCC  launched  a  courseware  development 
effort  to  revise  portions  of  the  demonstration  courseware  and  to  create  new 
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course  materials.     NVCC  is  experimenting  with  several  new  applications  of  the 
TICCIT  system,  including  a  biology  testing  program,  an  adjunctive  English 
grammar  course,  and  chemistry  and  poetry  course  modules.     NVCC  will  also 
be  developing  a  hew  math  course  for  science  students  xrnder  the  National 
Science  Foundation  CAUSE  and  Local  Assessment  grants.     NVCC  will  also  make 
major  hardware  improvements  to  their  system,  with  the  a^dit^e^of  memory,  a 
new  disk  drive  and  character  generator. 

Phoenix  College,     The  TICCIT  system  at  Phoenix  College  has  been  fully, 
integrated  into  the  freshman  English  program,  delivering  both  remedial  and 
freshman  composition  courses.     In  addition  to  its  role  in  the  English  program, 
the  TICCIT  system  is  being  used  to  deliver  instruction  in  remedial  math,  fire 
science,  and  medical  terminology. 

Phoenix  College  is  currently  exploring  the  possibilities  of  expanding 
its  rTCC:T  system  through  the  addition  of  a  new  disk  drive.  ^n^addition, 
Maricopa  C.mnty  Community  College  District  is  considering  expanding  the  use 
of  TICCIT  instruction  to  several  other  o"f  their  college  campuses. 

9.1.  L    Current  System  Developments 

Improvements  in  and  enhancements  to  the  TICCIT  system  have  occurred 
continuously  since  the  beginning  of  the  demonstration  period  in  1974.  Some 
of  the  modifications  we*e  designed  to  correct  specific  problems  encountered 
during  the  demonstration  period,  while  others  constitute  new  features  that 
should  broaden  TICCIT' s  capabilities. 

As  a  result  oE.the  demonstration  period,  the  TICCIT  test  model  was 
expanded,   the  advisor  program  revised  and  inter-communication  features  added. 
Because  the  original  "tailored  testing"  model  proved  too  restrictive  to 
courseware  authors,  a  second  testing  method  was  developed  in  order  to  allow 
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the  administration  of  tests  with  a  fixed  number  of  items  and  passing  percentages 
Although  the  "tailored  testing"  model  is  still  used  for  most  tests,  the  new 
fixed-length  model  expands  authoring  alternatives  and  greatly  simplifies  many 
applications  of  TICCIT.     In  addition,  the  Advisor  program  has  been  modified 
to  improve  the  quality  of  advice  given  by,  the  system.     Finally,  because  of  the 
need  to  get  in  touch  with  students  working  outside  of  classroom  hours  and  the 
need  to  locate  and  communicate  with  students  at  work,  a  number  of  communications 
features  were  added  to  allow  each  user  to  pass  messages  via  "mailboxes"  and 
to  allow  inter-terminal  communications .     Using  "mailbox",  an  instructor  can 
leave  messages,  information,  content  or  schedule  comments  for  students,  and 
students  can  leave  "mail"  for  other  students  as  well  as  for  the  instructor. 
The  inter-terminal  communications  system  makes  it  possible  to  any  user  to 
communicate  in  real-time  with  any  other  user  on  the  system. 

Continued  development  of  the  TICCIT  system  by  Hazeltine  has  resulted 
also  tin  several  major  advances.  Improvements  have  been  made  in  all  major 
TICCIT  subsystems,  including  hardware,  software,  and  courseware. 

Hardware  improvements  in  the  computer  and  display  generation  subsystems 
have  resulted  in  lower  purchase  and  maintenance  costs  and  increased  system 
reliability.     Because  of  recent  technological  advances,   the  TICCIT  system  now 
consists  of  fewer  hardware  components  yet  performs  more  efficiently  than 
previous  versions  of  the  system.     The  computer  subsystem  now  uses  the  most 
advanced  minicomputer  and  mass  menjory  devices,  making  it  possible  to  support 

a 

a  complete  TICCIT  system  using  only  one  minicomputer  rather  than  the  two 
required  previously.     In  addition,   the  moving  head  disk  storage  capacity  has 
increased  substantially,  allowing  a  75%  decrease  in  the  number  of  disks  needed 
per  system. 
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The  display  generation  subsystem  has  been  updated  with  state-of-the-art 
solid  state  memory  devices.     As  a  result,  the  new  display  refresh  memory  uses 
approximately  70%  fewer  printed  circuit  boards  than  previous  TICCIT  systems. 
This  reduction  in  the  number  of  components  and" interconnections  has  been  a 
major  factor  in  reducing  purchase  and  maintenance  costs  and  increasing  system 
reliability.     Additional  improvements  in  the  circuit  designtof  the  TICCIT 
student  terminal  have  produced  an  even  better  TICCIT  dispti  quality  than 
was  previously  possible. 

The  TICCIT  software  system  has  also  been  improved.     The  operating  systems 
originally  developed  for  the  TICCIT  system  have  been  revised,  greatly  increasing 
the  flexibility  and  reliability  of  the  system.     This  new  software  facilitates 
a  single-computer  system  configuration,  flexibility  in  the  use  of  disk  space, 
and  the  addition  of  special-purpose  devices  to  the  system. 

A  new  courseware  authoring  facility  called  TAL  (TICCIT  Authoring  Language) 
has  greatly  improved  the  flexibility  afforded  TICCIT  authors.     TAL  complements 
the  original  highly  structured  authoring  procedure  and  provides  a  general 
purpose  authoring  capability  chat  takes  full  advantage  of  existing  TICCIT 
display  construction  and  response  analysis  features.     TAL  has  been  specifically 
designed  to  expedite  the  authoring  of  many  different  types  of  instruction, 
including  simulations,  interactions  with  training  devices,  games,  and  drills. 

A  recent  addition  to  the  TICCIT  program  is  a  self-contained,  four 
terminal,  transportable  system.     Hazeltine  is  using  this  system  to  demonstrate 
major  TICCIT  capabilities  at  national  conventions  and  trade  shows.  Marketing 
representatives  are  also  showing  the  unit  at  potential  users'  facilities, 
demonstrating  with  the  users'  own  materials  how  a  TICCIT  system  might  be  used 
in  their  training  program. 
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9.2.e    The  Future  of  TICCIT 

The  use  of  minicomputer  and  television  technology  and  the  advances 
in  iHdeo  disk  and  microcomputer  technology  will  play  important  roles  in  the 

r 

future  of  the  TICCIT  system.     Video  disk  technology  will  greatly  improve  the 
system's  ability  to  present  pictorial  information  and  rapidly  access  instructional 
TV  segments  as  part  of  complete  courses  of  instruction.     The  video  disk  in 
conjunction  with  microcomputer  technology  opens  the  way  for  TICCIT  applications 
in  locations  where  one,  or  only  a  few  terminals  are  needed.     For  these  terminals, 
there  will  be  no  requirement,  for  telephone  line  connections  to  a  remote^computer . 

The  economic  benefits  of  the  TICCIT  system  will  also  "continue  to  increase. 
The  reduction  in  cost  and  improvement  in  capabilities  of  the  components  that 
make  up  the  TICCIT  system  will  lead  to  successive  improvements  in  the 
system's  overall  cost  effectiveness. 

Since  its  original  demonstration,  the  TICCIT  system  has  been  an  extremely 
valuable  resource  for  improving  instruction.     Building  upon  that  record,  the 
system  will  continue  to  provide  the  foundation  for  successful  training  and 
educational  programs.  ^ 
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